[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:14 dist-test-slave-ss4n krb5kdc[23114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:14 dist-test-slave-ss4n krb5kdc[23114](info): set up 2 sockets
Aug 11 04:41:14 dist-test-slave-ss4n krb5kdc[23114](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 04:41:15.728413 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:15 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:15.875205 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44055
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:44055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:16.036931 23129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:16.037228 23129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:16.037271 23129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:16.041762 23129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:16.041841 23129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:16.041862 23129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:16.041880 23129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:16.041896 23129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:16.047690 23129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:44055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:16.048804 23129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:16.049935 23129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:16.060252 23137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.060271 23135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:16.060459 23129 server_base.cc:1034] running on GCE node
W20260811 04:41:16.060271 23134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:16.061259 23129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:16.062530 23129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:16.063745 23129 hybrid_clock.cc:648] HybridClock initialized: now 1786423276063735 us; error 77 us; skew 500 ppm
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:16.067231 23129 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:16.068167 23129 webserver.cc:460] Webserver started at http://127.22.144.190:42137/ using document root <none> and password file <none>
I20260811 04:41:16.068747 23129 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:16.068810 23129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:16.069036 23129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:16.071560 23129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "e78a85a1bae84bb8be10f2e17477cf6a"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "b1c147f5991f4841f2f74103ce45c0ff"
server_key_iv: "6ee4dbab99f634c67156a32f5d58a044"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.072216 23129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "e78a85a1bae84bb8be10f2e17477cf6a"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "b1c147f5991f4841f2f74103ce45c0ff"
server_key_iv: "6ee4dbab99f634c67156a32f5d58a044"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.076284 23129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 04:41:16.079033 23144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.080577 23129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:16.080725 23129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "e78a85a1bae84bb8be10f2e17477cf6a"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "b1c147f5991f4841f2f74103ce45c0ff"
server_key_iv: "6ee4dbab99f634c67156a32f5d58a044"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.080847 23129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:16.096449 23129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:16.100229 23129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:16.100488 23129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:16.114923 23129 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:44055
I20260811 04:41:16.114941 23199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:44055 every 8 connection(s)
I20260811 04:41:16.116091 23129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:16.119377 23200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:16.125088 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23129
I20260811 04:41:16.125288 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:16.126423 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a: Bootstrap starting.
I20260811 04:41:16.127182 23106 external_mini_cluster.cc:1428] Setting key 9beb6ddfb335626bd8dd6b29e46fead5
I20260811 04:41:16.129917 23200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:16.130861 23200 log.cc:824] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:16.133154 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a: No bootstrap required, opened a new log
I20260811 04:41:16.137812 23200 raft_consensus.cc:348] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44055 } }
I20260811 04:41:16.138128 23200 raft_consensus.cc:374] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:16.138252 23200 raft_consensus.cc:729] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78a85a1bae84bb8be10f2e17477cf6a, State: Initialized, Role: FOLLOWER
I20260811 04:41:16.138785 23200 consensus_queue.cc:260] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44055 } }
I20260811 04:41:16.138963 23200 raft_consensus.cc:388] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:16.139057 23200 raft_consensus.cc:482] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:16.139190 23200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:16.140234 23200 raft_consensus.cc:504] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44055 } }
I20260811 04:41:16.140623 23200 leader_election.cc:302] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e78a85a1bae84bb8be10f2e17477cf6a; no voters:
I20260811 04:41:16.140997 23200 leader_election.cc:288] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:16.141088 23205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:16.141330 23205 raft_consensus.cc:686] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [term 1 LEADER]: Becoming Leader. State: Replica: e78a85a1bae84bb8be10f2e17477cf6a, State: Running, Role: LEADER
I20260811 04:41:16.141742 23205 consensus_queue.cc:237] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44055 } }
I20260811 04:41:16.142529 23200 sys_catalog.cc:564] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:16.144090 23206 sys_catalog.cc:455] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e78a85a1bae84bb8be10f2e17477cf6a. Latest consensus state: current_term: 1 leader_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44055 } } }
I20260811 04:41:16.144251 23206 sys_catalog.cc:458] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:16.144644 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e78a85a1bae84bb8be10f2e17477cf6a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44055 } } }
I20260811 04:41:16.144881 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:16.146607 23215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:16.150652 23215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:16.158020 23215 catalog_manager.cc:1347] Generated new cluster ID: d7784f3db2264a739a46da3947a142c8
I20260811 04:41:16.158100 23215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:16.161185 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:16.131128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.22.144.190:44055)
0811 04:41:16.131666 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:16.131695 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:16.133439 (+ 1744us) server_negotiation.cc:407] Connection header received
0811 04:41:16.137294 (+ 3855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:16.137314 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:16.137774 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:16.138189 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:16.141361 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.143270 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.145287 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.147156 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.147321 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:16.154042 (+ 6721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:16.154049 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:16.154065 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:16.154100 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:16.156678 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:16.157581 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:16.157590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:16.157595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:16.157744 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:16.158294 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:16.158299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:16.158301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:16.158834 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:16.159096 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:16.159798 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 04:41:16.160091 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":151,"threads_started":1}
I20260811 04:41:16.182564 23215 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:16.183816 23215 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:16.188912 23215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e78a85a1bae84bb8be10f2e17477cf6a: Generated new TSK 0
I20260811 04:41:16.189685 23215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:16.233850 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44055
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:16.380672 23230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:16.380971 23230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:16.381018 23230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:16.385717 23230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:16.385787 23230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:16.385872 23230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:16.391566 23230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44055
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:16.392802 23230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:16.393985 23230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:16.404414 23238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.404505 23240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.404527 23237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:16.404814 23230 server_base.cc:1034] running on GCE node
I20260811 04:41:16.405321 23230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:16.406101 23230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:16.407302 23230 hybrid_clock.cc:648] HybridClock initialized: now 1786423276407284 us; error 32 us; skew 500 ppm
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:16.410373 23230 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:16.411366 23230 webserver.cc:460] Webserver started at http://127.22.144.129:36411/ using document root <none> and password file <none>
I20260811 04:41:16.412024 23230 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:16.412135 23230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:16.412379 23230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:16.414760 23230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "8375c6138e8d439d9b3fabf49f50e31c"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "f85249d2ab1d28d282718f28a2fdf6ef"
server_key_iv: "3386f6eb2efc74a3c8441f1539b098c3"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.415401 23230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "8375c6138e8d439d9b3fabf49f50e31c"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "f85249d2ab1d28d282718f28a2fdf6ef"
server_key_iv: "3386f6eb2efc74a3c8441f1539b098c3"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.419797 23230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:16.422613 23247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.423951 23230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:16.424096 23230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "8375c6138e8d439d9b3fabf49f50e31c"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "f85249d2ab1d28d282718f28a2fdf6ef"
server_key_iv: "3386f6eb2efc74a3c8441f1539b098c3"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.424226 23230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:16.453888 23230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:16.457634 23230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:16.457861 23230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:16.458544 23230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:16.459439 23230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:16.459515 23230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.459614 23230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:16.459666 23230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.470321 23230 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:40975
I20260811 04:41:16.470474 23369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:40975 every 8 connection(s)
I20260811 04:41:16.471285 23230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:16.472900 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23230
I20260811 04:41:16.473012 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:16.473371 23106 external_mini_cluster.cc:1428] Setting key d27863f8813702f8a85ba50288d7dcc5
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:16.486827 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:16.473593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:59179 (local address 127.22.144.190:44055)
0811 04:41:16.473745 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:16.473751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:16.474628 (+ 877us) server_negotiation.cc:407] Connection header received
0811 04:41:16.476033 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:16.476037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:16.476089 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:16.476169 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:16.477515 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.478051 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.479339 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.479627 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.479735 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:16.482794 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:16.482797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:16.482800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:16.482824 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:16.484456 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:16.485036 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:16.485039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:16.485041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:16.485128 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:16.485582 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:16.485587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:16.485589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:16.485780 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:16.485896 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:16.486474 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 04:41:16.486655 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:16.487794 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:16.473973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44055 (local address 127.22.144.129:59179)
0811 04:41:16.474456 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:16.474511 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:16.475807 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:16.476371 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:16.476377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:16.476741 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:16.477317 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:16.477333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.478188 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.478192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:16.479200 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:16.479211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.479823 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.479830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:16.480077 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:16.480768 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:16.480793 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:16.482607 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:16.484608 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:16.484615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:16.484631 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:16.484923 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:16.485275 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:16.485278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:16.485280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:16.485430 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:16.485895 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:16.485903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:16.486167 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 04:41:16.486379 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 04:41:16.486726 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":147,"threads_started":1}
I20260811 04:41:16.489131 23370 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44055
I20260811 04:41:16.489411 23370 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:16.489984 23370 heartbeater.cc:511] Master 127.22.144.190:44055 requested a full tablet report, sending...
I20260811 04:41:16.491632 23163 ts_manager.cc:194] Registered new tserver with Master: 8375c6138e8d439d9b3fabf49f50e31c (127.22.144.129:40975)
I20260811 04:41:16.492799 23163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:59179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:16.508586 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44055
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:16.673164 23375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:16.673559 23375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:16.673653 23375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:16.678066 23375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:16.678130 23375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:16.678228 23375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:16.683915 23375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44055
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:16.685128 23375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:16.686326 23375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:16.697449 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.697715 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.697772 23381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:16.698002 23375 server_base.cc:1034] running on GCE node
I20260811 04:41:16.698535 23375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:16.699285 23375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:16.700487 23375 hybrid_clock.cc:648] HybridClock initialized: now 1786423276700474 us; error 41 us; skew 500 ppm
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:16.703985 23375 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:16.705019 23375 webserver.cc:460] Webserver started at http://127.22.144.130:38775/ using document root <none> and password file <none>
I20260811 04:41:16.705684 23375 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:16.705781 23375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:16.706020 23375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:16.708442 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "dc9367158fe74fba8d694f3334f17e3f"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "6016d8034676b83ef7d8856842800495"
server_key_iv: "712a2c26e1effb091c1a1a860cbf296d"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.709098 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "dc9367158fe74fba8d694f3334f17e3f"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "6016d8034676b83ef7d8856842800495"
server_key_iv: "712a2c26e1effb091c1a1a860cbf296d"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.713411 23375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 04:41:16.716368 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.717994 23375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:16.718168 23375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "dc9367158fe74fba8d694f3334f17e3f"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "6016d8034676b83ef7d8856842800495"
server_key_iv: "712a2c26e1effb091c1a1a860cbf296d"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.718308 23375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:16.741947 23375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:16.745972 23375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:16.746236 23375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:16.747056 23375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:16.748108 23375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:16.748193 23375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.748265 23375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:16.748301 23375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.769038 23375 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:44251
I20260811 04:41:16.769057 23514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:44251 every 8 connection(s)
I20260811 04:41:16.770123 23375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:16.778076 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23375
I20260811 04:41:16.778172 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:16.778430 23106 external_mini_cluster.cc:1428] Setting key 4a3cf2296c5c9214ddf2af4268aa2ebf
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:16.788901 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:16.772321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:60857 (local address 127.22.144.190:44055)
0811 04:41:16.772467 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:16.772471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:16.774171 (+ 1700us) server_negotiation.cc:407] Connection header received
0811 04:41:16.775580 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:16.775584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:16.775662 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:16.775798 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:16.777624 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.778335 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.780218 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.780502 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.780642 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:16.784457 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:16.784463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:16.784466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:16.784503 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:16.786558 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:16.787120 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:16.787126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:16.787130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:16.787237 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:16.787612 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:16.787618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:16.787621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:16.787901 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:16.788018 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:16.788544 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 04:41:16.788696 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 04:41:16.789830 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:16.772721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44055 (local address 127.22.144.130:60857)
0811 04:41:16.773964 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:16.774018 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:16.775296 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:16.776011 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:16.776019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:16.776694 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:16.777393 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:16.777412 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.778498 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.778503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:16.780065 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:16.780077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:16.780687 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:16.780692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:16.780955 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:16.781763 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:16.781782 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:16.784235 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:16.786708 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:16.786713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:16.786726 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:16.786994 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:16.787344 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:16.787346 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:16.787348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:16.787490 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:16.788012 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:16.788017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:16.788280 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 04:41:16.788437 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 04:41:16.788712 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":711,"threads_started":1}
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:41:16.791397 23515 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44055
I20260811 04:41:16.791697 23515 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:16.792276 23515 heartbeater.cc:511] Master 127.22.144.190:44055 requested a full tablet report, sending...
I20260811 04:41:16.793430 23163 ts_manager.cc:194] Registered new tserver with Master: dc9367158fe74fba8d694f3334f17e3f (127.22.144.130:44251)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:41:16.794348 23163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:60857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:16.813649 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44055
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:16.953007 23522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:16.953303 23522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:16.953351 23522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:16.957756 23522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:16.957837 23522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:16.957904 23522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:16.963747 23522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44055
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:16.964917 23522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:16.966014 23522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:16.978365 23531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.978456 23529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:16.978585 23527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:16.978847 23522 server_base.cc:1034] running on GCE node
I20260811 04:41:16.979301 23522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:16.979974 23522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:16.981207 23522 hybrid_clock.cc:648] HybridClock initialized: now 1786423276981185 us; error 35 us; skew 500 ppm
Aug 11 04:41:16 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:16.984920 23522 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:16.986069 23522 webserver.cc:460] Webserver started at http://127.22.144.131:36301/ using document root <none> and password file <none>
I20260811 04:41:16.986711 23522 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:16.986783 23522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:16.986965 23522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:16.989428 23522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "de79441e466148dc9bb18e4fbc2c2a8c"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "b2b3df2e36e21b10dbad50e7c08a0595"
server_key_iv: "8a5216a561328a34f3e8d57f63e6d0e5"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.990141 23522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "de79441e466148dc9bb18e4fbc2c2a8c"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "b2b3df2e36e21b10dbad50e7c08a0595"
server_key_iv: "8a5216a561328a34f3e8d57f63e6d0e5"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.994952 23522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:16.997910 23538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:16.999441 23522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:16.999624 23522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "de79441e466148dc9bb18e4fbc2c2a8c"
format_stamp: "Formatted at 2026-08-11 04:41:16 on dist-test-slave-ss4n"
server_key: "b2b3df2e36e21b10dbad50e7c08a0595"
server_key_iv: "8a5216a561328a34f3e8d57f63e6d0e5"
server_key_version: "encryptionkey@0"
I20260811 04:41:16.999759 23522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:17.014240 23522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:17.018496 23522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:17.018795 23522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:17.019706 23522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:17.020759 23522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:17.020843 23522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:17.020917 23522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:17.020975 23522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:17.049018 23522 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:44115
I20260811 04:41:17.049139 23659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:44115 every 8 connection(s)
I20260811 04:41:17.050172 23522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:17.052435 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23522
I20260811 04:41:17.052619 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:17.052932 23106 external_mini_cluster.cc:1428] Setting key 9899f5041cc8313af1877acdeaa02fbf
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:17.067689 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:17.052509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:37999 (local address 127.22.144.190:44055)
0811 04:41:17.052695 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:17.052700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:17.054070 (+ 1370us) server_negotiation.cc:407] Connection header received
0811 04:41:17.056027 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:17.056031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:17.056111 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:17.056187 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:17.057806 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.058415 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.059814 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.060049 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.060137 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:17.063475 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:17.063479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:17.063481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:17.063508 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:17.065107 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:17.065813 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:17.065816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:17.065817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:17.065944 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:17.066342 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:17.066346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:17.066350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:17.066597 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:17.066698 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:17.067361 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 04:41:17.067515 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 04:41:17.068637 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:17.052976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44055 (local address 127.22.144.131:37999)
0811 04:41:17.053885 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:17.053931 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:17.055784 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:17.056365 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:17.056372 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:17.056893 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:17.057638 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:17.057655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.058591 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.058595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:17.059668 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:17.059680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.060222 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.060225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:17.060436 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:17.061193 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:17.061211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:17.063290 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:17.065228 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:17.065234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:17.065248 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:17.065711 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:17.066062 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:17.066065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:17.066066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:17.066218 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:17.066713 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:17.066719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:17.067035 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 04:41:17.067252 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 04:41:17.067665 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":294,"threads_started":1}
I20260811 04:41:17.069875 23660 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44055
I20260811 04:41:17.070169 23660 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:17.070696 23660 heartbeater.cc:511] Master 127.22.144.190:44055 requested a full tablet report, sending...
I20260811 04:41:17.071758 23163 ts_manager.cc:194] Registered new tserver with Master: de79441e466148dc9bb18e4fbc2c2a8c (127.22.144.131:44115)
I20260811 04:41:17.072338 23163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:37999
I20260811 04:41:17.080220 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423277, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423277, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:17.113464 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:17.102260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54786 (local address 127.22.144.129:40975)
0811 04:41:17.102615 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:17.102621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:17.102647 (+ 26us) server_negotiation.cc:407] Connection header received
0811 04:41:17.102702 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:17.102708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:17.102898 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:17.103033 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:17.103934 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.104972 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.106049 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.106543 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.106645 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:17.109427 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:17.109436 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:17.109451 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:17.109529 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:17.111515 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:17.111996 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:17.112004 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:17.112005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:17.112101 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:17.112369 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:17.112371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:17.112372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:17.112696 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:17.112865 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:17.113072 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 04:41:17.113281 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
W20260811 04:41:17.114642 23278 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54786
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:17.147444 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:17.138212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54802 (local address 127.22.144.129:40975)
0811 04:41:17.138394 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:17.138399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:17.138554 (+ 155us) server_negotiation.cc:407] Connection header received
0811 04:41:17.138754 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:17.138758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:17.138804 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:17.138895 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:17.139649 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.140188 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.141096 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:17.141334 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:17.141450 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:17.144226 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:17.144229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:17.144231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:17.144259 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:17.145709 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:17.146140 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:17.146144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:17.146145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:17.146229 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:17.146511 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:17.146517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:17.146522 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:17.146710 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:17.146857 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:17.147114 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 04:41:17.147270 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 04:41:17.148794 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23230
I20260811 04:41:17.157217 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23375
I20260811 04:41:17.165642 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23522
I20260811 04:41:17.173454 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23129
2026-08-11T04:41:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2635 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23677](info): set up 2 sockets
Aug 11 04:41:17 dist-test-slave-ss4n krb5kdc[23677](info): commencing operation
krb5kdc: starting...
W20260811 04:41:18.400949 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:18 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:18.541633 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44347
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:44347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:18.661607 23690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:18.661959 23690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:18.662051 23690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:18.666306 23690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:18.666400 23690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:18.666440 23690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:18.666494 23690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:18.666534 23690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:18.672011 23690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:44347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:18.673156 23690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:18.674281 23690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:18.683384 23695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:18.683656 23690 server_base.cc:1034] running on GCE node
W20260811 04:41:18.683367 23698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:18.683390 23696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:18.684461 23690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:18.685720 23690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:18.686950 23690 hybrid_clock.cc:648] HybridClock initialized: now 1786423278686915 us; error 48 us; skew 500 ppm
Aug 11 04:41:18 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:18.690210 23690 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:18.691187 23690 webserver.cc:460] Webserver started at http://127.22.144.190:33713/ using document root <none> and password file <none>
I20260811 04:41:18.691787 23690 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:18.691892 23690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:18.692160 23690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:18.694445 23690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "af44dce452bd454ebb5c4683a5029674"
format_stamp: "Formatted at 2026-08-11 04:41:18 on dist-test-slave-ss4n"
server_key: "2d970b548e7d541d7bd79b84d6845ab0"
server_key_iv: "116b96b730cf4447c2690134b382fe85"
server_key_version: "encryptionkey@0"
I20260811 04:41:18.695178 23690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "af44dce452bd454ebb5c4683a5029674"
format_stamp: "Formatted at 2026-08-11 04:41:18 on dist-test-slave-ss4n"
server_key: "2d970b548e7d541d7bd79b84d6845ab0"
server_key_iv: "116b96b730cf4447c2690134b382fe85"
server_key_version: "encryptionkey@0"
I20260811 04:41:18.699532 23690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:18.702307 23707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:18.703778 23690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 04:41:18.703946 23690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "af44dce452bd454ebb5c4683a5029674"
format_stamp: "Formatted at 2026-08-11 04:41:18 on dist-test-slave-ss4n"
server_key: "2d970b548e7d541d7bd79b84d6845ab0"
server_key_iv: "116b96b730cf4447c2690134b382fe85"
server_key_version: "encryptionkey@0"
I20260811 04:41:18.704082 23690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:18.716392 23690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:18.720170 23690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:18.720434 23690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:18.728421 23772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:44347 every 8 connection(s)
I20260811 04:41:18.728410 23690 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:44347
I20260811 04:41:18.729606 23690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:18.732482 23773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:18.738121 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674: Bootstrap starting.
I20260811 04:41:18.739573 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23690
I20260811 04:41:18.739688 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:18.739943 23106 external_mini_cluster.cc:1428] Setting key 07bd217ea4577e3751fdb1aefcae709a
I20260811 04:41:18.741056 23773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:18.741953 23773 log.cc:824] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:18.744217 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674: No bootstrap required, opened a new log
I20260811 04:41:18.748811 23773 raft_consensus.cc:348] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af44dce452bd454ebb5c4683a5029674" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44347 } }
I20260811 04:41:18.749121 23773 raft_consensus.cc:374] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:18.749197 23773 raft_consensus.cc:729] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af44dce452bd454ebb5c4683a5029674, State: Initialized, Role: FOLLOWER
Aug 11 04:41:18 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:18.749730 23773 consensus_queue.cc:260] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af44dce452bd454ebb5c4683a5029674" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44347 } }
I20260811 04:41:18.749923 23773 raft_consensus.cc:388] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:18.750001 23773 raft_consensus.cc:482] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:18.750113 23773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:18.751127 23773 raft_consensus.cc:504] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af44dce452bd454ebb5c4683a5029674" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44347 } }
I20260811 04:41:18.751539 23773 leader_election.cc:302] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af44dce452bd454ebb5c4683a5029674; no voters:
I20260811 04:41:18.751931 23773 leader_election.cc:288] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:18.752048 23780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:18.752323 23780 raft_consensus.cc:686] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [term 1 LEADER]: Becoming Leader. State: Replica: af44dce452bd454ebb5c4683a5029674, State: Running, Role: LEADER
I20260811 04:41:18.752657 23780 consensus_queue.cc:237] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af44dce452bd454ebb5c4683a5029674" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44347 } }
I20260811 04:41:18.753436 23773 sys_catalog.cc:564] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:18.754621 23781 sys_catalog.cc:455] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af44dce452bd454ebb5c4683a5029674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af44dce452bd454ebb5c4683a5029674" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44347 } } }
I20260811 04:41:18.754770 23781 sys_catalog.cc:458] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:18.755110 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af44dce452bd454ebb5c4683a5029674. Latest consensus state: current_term: 1 leader_uuid: "af44dce452bd454ebb5c4683a5029674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af44dce452bd454ebb5c4683a5029674" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44347 } } }
I20260811 04:41:18.755223 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:18.757006 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:18.741225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56160 (local address 127.22.144.190:44347)
0811 04:41:18.741726 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:18.741743 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:18.741779 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:41:18.742518 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:18.742538 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:18.742877 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:18.743164 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:18.744060 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:18.745115 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:18.746347 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:18.746843 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:18.746972 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:18.750266 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:18.750272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:18.750287 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:18.750318 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:18.752466 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:18.753030 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:18.753036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:18.753043 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:18.753206 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:18.753680 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:18.753685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:18.753687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:18.754343 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:18.755472 (+ 1129us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:18.755661 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 04:41:18.756010 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":306,"thread_start_us":183,"threads_started":1}
W20260811 04:41:18.758786 23797 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:18.758888 23797 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:18.758982 23789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:18.759920 23789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:18.765506 23789 catalog_manager.cc:1347] Generated new cluster ID: cfe79cd3e7cd418791b62775ea10a7d2
I20260811 04:41:18.765583 23789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:18.782240 23789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:18.783340 23789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:18.789043 23789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af44dce452bd454ebb5c4683a5029674: Generated new TSK 0
I20260811 04:41:18.789839 23789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:18.829023 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44347
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:18.949983 23802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:18.950273 23802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:18.950320 23802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:18.954763 23802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:18.954830 23802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:18.954939 23802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:18.960537 23802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:18.961812 23802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:18.962955 23802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:18.973295 23807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:18.973302 23810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:18.973528 23808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:18.973734 23802 server_base.cc:1034] running on GCE node
I20260811 04:41:18.974229 23802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:18.974887 23802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:18.976083 23802 hybrid_clock.cc:648] HybridClock initialized: now 1786423278976048 us; error 47 us; skew 500 ppm
Aug 11 04:41:18 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:18.979138 23802 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:18.980127 23802 webserver.cc:460] Webserver started at http://127.22.144.129:39309/ using document root <none> and password file <none>
I20260811 04:41:18.980716 23802 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:18.980814 23802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:18.981058 23802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:18.983351 23802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "7267a5ca183941b2a0272d40c3a10153"
format_stamp: "Formatted at 2026-08-11 04:41:18 on dist-test-slave-ss4n"
server_key: "0172f6ce359e451919c023f255013686"
server_key_iv: "8f4787d5c8eff8c7db2a8e79b3f9a382"
server_key_version: "encryptionkey@0"
I20260811 04:41:18.983992 23802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "7267a5ca183941b2a0272d40c3a10153"
format_stamp: "Formatted at 2026-08-11 04:41:18 on dist-test-slave-ss4n"
server_key: "0172f6ce359e451919c023f255013686"
server_key_iv: "8f4787d5c8eff8c7db2a8e79b3f9a382"
server_key_version: "encryptionkey@0"
I20260811 04:41:18.987934 23802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 04:41:18.990628 23819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:18.992003 23802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:18.992172 23802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "7267a5ca183941b2a0272d40c3a10153"
format_stamp: "Formatted at 2026-08-11 04:41:18 on dist-test-slave-ss4n"
server_key: "0172f6ce359e451919c023f255013686"
server_key_iv: "8f4787d5c8eff8c7db2a8e79b3f9a382"
server_key_version: "encryptionkey@0"
I20260811 04:41:18.992295 23802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:19.003901 23802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:19.007629 23802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:19.007903 23802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:19.008561 23802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:19.009528 23802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:19.009585 23802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.009657 23802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:19.009691 23802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.019100 23802 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:39705
I20260811 04:41:19.019186 23949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:39705 every 8 connection(s)
I20260811 04:41:19.020072 23802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:19.026746 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23802
I20260811 04:41:19.026857 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:19.027163 23106 external_mini_cluster.cc:1428] Setting key 2b58dce41fb46f3333ea09d87f2b1cac
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:19.036454 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.022211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:54873 (local address 127.22.144.190:44347)
0811 04:41:19.022374 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.022379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.023294 (+ 915us) server_negotiation.cc:407] Connection header received
0811 04:41:19.024667 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.024670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.024718 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.024797 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.026161 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.026901 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.028529 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.028836 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.028927 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.032304 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.032307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.032309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.032332 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.034044 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.034677 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.034680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.034681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.034760 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.035219 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.035222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.035223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.035402 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.035484 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.036209 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.036322 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 04:41:19.037760 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.022646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44347 (local address 127.22.144.129:54873)
0811 04:41:19.023127 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.023172 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.024449 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.024968 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.024974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.025347 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:19.026023 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.026038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.027068 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.027072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.028313 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.028324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.029005 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.029013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.029254 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.029945 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:19.029971 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:19.032124 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:19.034175 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:19.034183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:19.034203 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:19.034577 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:19.034878 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:19.034884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:19.034889 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:19.035114 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:19.035521 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:19.035530 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:19.035908 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.036177 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.036568 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:19.039160 23950 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44347
I20260811 04:41:19.039496 23950 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:19.040155 23950 heartbeater.cc:511] Master 127.22.144.190:44347 requested a full tablet report, sending...
I20260811 04:41:19.042097 23727 ts_manager.cc:194] Registered new tserver with Master: 7267a5ca183941b2a0272d40c3a10153 (127.22.144.129:39705)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:19.043445 23727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:54873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:19.062071 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44347
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:19.181449 23956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:19.181740 23956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:19.181833 23956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:19.185886 23956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:19.185947 23956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:19.186080 23956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:19.191612 23956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:19.192715 23956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:19.193961 23956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:19.204615 23962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:19.204644 23964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:19.204633 23961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:19.204699 23956 server_base.cc:1034] running on GCE node
I20260811 04:41:19.205319 23956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:19.206192 23956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:19.207417 23956 hybrid_clock.cc:648] HybridClock initialized: now 1786423279207397 us; error 32 us; skew 500 ppm
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:19.210621 23956 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:19.211583 23956 webserver.cc:460] Webserver started at http://127.22.144.130:35641/ using document root <none> and password file <none>
I20260811 04:41:19.212244 23956 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:19.212357 23956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:19.212620 23956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:19.215023 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "c1b61460988d49e9b2ab65b6e986fe8d"
format_stamp: "Formatted at 2026-08-11 04:41:19 on dist-test-slave-ss4n"
server_key: "3ec5f3b44ed87ec1af8b15c248ad3665"
server_key_iv: "a1cb15d6930af821755b86cadee8ee4b"
server_key_version: "encryptionkey@0"
I20260811 04:41:19.215660 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "c1b61460988d49e9b2ab65b6e986fe8d"
format_stamp: "Formatted at 2026-08-11 04:41:19 on dist-test-slave-ss4n"
server_key: "3ec5f3b44ed87ec1af8b15c248ad3665"
server_key_iv: "a1cb15d6930af821755b86cadee8ee4b"
server_key_version: "encryptionkey@0"
I20260811 04:41:19.220033 23956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:19.222990 23972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.224426 23956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:19.224581 23956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "c1b61460988d49e9b2ab65b6e986fe8d"
format_stamp: "Formatted at 2026-08-11 04:41:19 on dist-test-slave-ss4n"
server_key: "3ec5f3b44ed87ec1af8b15c248ad3665"
server_key_iv: "a1cb15d6930af821755b86cadee8ee4b"
server_key_version: "encryptionkey@0"
I20260811 04:41:19.224716 23956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:19.244668 23956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:19.248286 23956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:19.248531 23956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:19.249235 23956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:19.250195 23956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:19.250276 23956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.250348 23956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:19.250401 23956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.260366 23956 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:33949
I20260811 04:41:19.260390 24096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:33949 every 8 connection(s)
I20260811 04:41:19.261340 23956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:19.270756 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 23956
I20260811 04:41:19.270915 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:19.271263 23106 external_mini_cluster.cc:1428] Setting key 14efd99e64f254eb85a13fe862871c4f
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:19.276767 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.263392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:60597 (local address 127.22.144.190:44347)
0811 04:41:19.263551 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.263556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.264499 (+ 943us) server_negotiation.cc:407] Connection header received
0811 04:41:19.265983 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.265986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.266039 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.266137 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.267423 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.267995 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.269204 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.269426 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.269605 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.272520 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.272523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.272525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.272551 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.274229 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.274918 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.274922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.274925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.275063 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.275490 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.275495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.275498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.275770 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.275885 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.276503 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.276640 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:41:19.278247 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.263838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44347 (local address 127.22.144.130:60597)
0811 04:41:19.264328 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.264375 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.265744 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.266300 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.266306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.266738 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:19.267259 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.267272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.268141 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.268144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.269085 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.269095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.269686 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.269689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.269844 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.270484 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:19.270501 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:19.272344 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:19.274353 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:19.274363 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:19.274383 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:19.274791 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:19.275204 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:19.275207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:19.275208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:19.275375 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:19.275893 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:19.275899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:19.276171 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.276428 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.276921 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":117,"threads_started":1}
I20260811 04:41:19.279695 24097 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44347
I20260811 04:41:19.279999 24097 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:19.280541 24097 heartbeater.cc:511] Master 127.22.144.190:44347 requested a full tablet report, sending...
I20260811 04:41:19.281744 23727 ts_manager.cc:194] Registered new tserver with Master: c1b61460988d49e9b2ab65b6e986fe8d (127.22.144.130:33949)
I20260811 04:41:19.282352 23727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:60597
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:19.305075 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44347
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:19.427323 24102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:19.427683 24102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:19.427771 24102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:19.432266 24102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:19.432386 24102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:19.432488 24102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:19.437958 24102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:19.439106 24102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:19.440181 24102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:19.450333 24107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:19.450398 24111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:19.450415 24102 server_base.cc:1034] running on GCE node
W20260811 04:41:19.450333 24109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:19.451176 24102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:19.451857 24102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:19.453035 24102 hybrid_clock.cc:648] HybridClock initialized: now 1786423279453014 us; error 35 us; skew 500 ppm
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:19.456336 24102 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:19.457294 24102 webserver.cc:460] Webserver started at http://127.22.144.131:37819/ using document root <none> and password file <none>
I20260811 04:41:19.457943 24102 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:19.458050 24102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:19.458292 24102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:19.460412 24102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "a7dd31310c10400780cf7ad643ead295"
format_stamp: "Formatted at 2026-08-11 04:41:19 on dist-test-slave-ss4n"
server_key: "c74026c40ca6ac4cdebf2ac98574fa15"
server_key_iv: "fb61792e3080878dd5620ebc822153cc"
server_key_version: "encryptionkey@0"
I20260811 04:41:19.461017 24102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "a7dd31310c10400780cf7ad643ead295"
format_stamp: "Formatted at 2026-08-11 04:41:19 on dist-test-slave-ss4n"
server_key: "c74026c40ca6ac4cdebf2ac98574fa15"
server_key_iv: "fb61792e3080878dd5620ebc822153cc"
server_key_version: "encryptionkey@0"
I20260811 04:41:19.465126 24102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:19.467907 24120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.469223 24102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:19.469367 24102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "a7dd31310c10400780cf7ad643ead295"
format_stamp: "Formatted at 2026-08-11 04:41:19 on dist-test-slave-ss4n"
server_key: "c74026c40ca6ac4cdebf2ac98574fa15"
server_key_iv: "fb61792e3080878dd5620ebc822153cc"
server_key_version: "encryptionkey@0"
I20260811 04:41:19.469550 24102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:19.484743 24102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:19.488171 24102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:19.488444 24102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:19.489161 24102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:19.490316 24102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:19.490397 24102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.490473 24102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:19.490530 24102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:19.501369 24102 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:35927
I20260811 04:41:19.501390 24243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:35927 every 8 connection(s)
I20260811 04:41:19.502452 24102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:19.503391 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 24102
I20260811 04:41:19.503537 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:19.503818 23106 external_mini_cluster.cc:1428] Setting key ed6a0cee268c8666f49500e3af5ed03f
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:19.517678 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.504477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:44245 (local address 127.22.144.190:44347)
0811 04:41:19.504685 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.504690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.505742 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 04:41:19.507093 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.507096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.507146 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.507221 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.508525 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.509222 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.510627 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.510884 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.511013 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.513919 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.513923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.513925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.513954 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.515394 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.515998 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.516002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.516003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.516094 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.516412 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.516414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.516415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.516639 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.516717 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.517271 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.517428 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 04:41:19.518635 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.504974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44347 (local address 127.22.144.131:44245)
0811 04:41:19.505560 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.505633 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.506890 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.507385 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.507392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.507797 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:19.508392 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.508409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.509390 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.509394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.510474 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.510485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.511062 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.511066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.511265 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.511876 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:19.511894 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:19.513739 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:19.515517 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:19.515523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:19.515536 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:19.515804 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:19.516183 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:19.516186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:19.516187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:19.516320 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:19.516753 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:19.516760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:19.517019 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.517192 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.517607 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":205,"threads_started":1}
I20260811 04:41:19.519913 24244 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44347
I20260811 04:41:19.520174 24244 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:19.520694 24244 heartbeater.cc:511] Master 127.22.144.190:44347 requested a full tablet report, sending...
I20260811 04:41:19.521796 23727 ts_manager.cc:194] Registered new tserver with Master: a7dd31310c10400780cf7ad643ead295 (127.22.144.131:35927)
I20260811 04:41:19.522392 23727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:44245
I20260811 04:41:19.530635 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:41:19.542052 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.534480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.22.144.190:44347)
0811 04:41:19.534669 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.534674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.534876 (+ 202us) server_negotiation.cc:407] Connection header received
0811 04:41:19.534975 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.534977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.535022 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.535088 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.535794 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.536318 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.537143 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.537357 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.537437 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.538893 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.538896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.538898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.538921 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.540416 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.540823 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.540825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.540826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.540900 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.541141 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.541143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.541144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.541359 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.541464 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.541712 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.541820 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 04:41:19.547349 23727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:19.550734 23727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:19.566701 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.560308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:33949 (local address 127.0.0.1:43588)
0811 04:41:19.561110 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.561167 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.561330 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.562235 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.562241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.562265 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:19.562416 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.562424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.563570 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.563577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.564923 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.564934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.566121 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.566124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.566236 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.566275 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.566390 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.566519 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":109,"threads_started":1}
I20260811 04:41:19.567200 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.561184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:35927 (local address 127.0.0.1:35154)
0811 04:41:19.561755 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.561770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.561881 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.562648 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.562651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.562671 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:19.562747 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.562752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.564380 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.564385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.565576 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.565590 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.566738 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.566742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.566840 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.566868 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.566966 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.567038 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":21,"thread_start_us":120,"threads_started":1}
I20260811 04:41:19.567512 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.560613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43588 (local address 127.22.144.130:33949)
0811 04:41:19.561138 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.561152 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.561476 (+ 324us) server_negotiation.cc:407] Connection header received
0811 04:41:19.561580 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.561596 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.561835 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.562003 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:19.562553 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.563431 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.565076 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.565792 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.565912 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.566054 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.566970 (+ 916us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.567356 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":119,"threads_started":1}
I20260811 04:41:19.567988 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.561694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.22.144.131:35927)
0811 04:41:19.562140 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.562150 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.562181 (+ 31us) server_negotiation.cc:407] Connection header received
0811 04:41:19.562244 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.562248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.562440 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.562652 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:19.562853 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.564221 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.565734 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.566602 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.566735 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.566882 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.567686 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.567809 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":159,"threads_started":1}
I20260811 04:41:19.568325 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.560308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39705 (local address 127.0.0.1:48236)
0811 04:41:19.561283 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.561296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.561367 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.562430 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.562434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.562463 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:19.562621 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.562630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.564314 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.564318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.566331 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.566342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.567550 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.567557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.568008 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.568031 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.568109 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.568179 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":95,"thread_start_us":113,"threads_started":1}
I20260811 04:41:19.568607 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.560596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.22.144.129:39705)
0811 04:41:19.561313 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.561323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.561354 (+ 31us) server_negotiation.cc:407] Connection header received
0811 04:41:19.562047 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.562056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.562271 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.562422 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:19.562771 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.564139 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.566503 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.567414 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.567564 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.567712 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.568343 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.568464 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":107,"threads_started":1}
I20260811 04:41:19.570782 24023 tablet_service.cc:1467] Processing CreateTablet for tablet 3310b09e22204cecb6deb181270e01b4 (DEFAULT_TABLE table=test-table [id=dc03ce69e7ec4c4788f4fa664ea91f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:19.571921 24023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3310b09e22204cecb6deb181270e01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:19.571563 24169 tablet_service.cc:1467] Processing CreateTablet for tablet 3310b09e22204cecb6deb181270e01b4 (DEFAULT_TABLE table=test-table [id=dc03ce69e7ec4c4788f4fa664ea91f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:19.571990 23874 tablet_service.cc:1467] Processing CreateTablet for tablet 3310b09e22204cecb6deb181270e01b4 (DEFAULT_TABLE table=test-table [id=dc03ce69e7ec4c4788f4fa664ea91f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:19.572706 24169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3310b09e22204cecb6deb181270e01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:19.573125 23874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3310b09e22204cecb6deb181270e01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:19.579144 24263 tablet_bootstrap.cc:492] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: Bootstrap starting.
I20260811 04:41:19.580945 24264 tablet_bootstrap.cc:492] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: Bootstrap starting.
I20260811 04:41:19.582264 24265 tablet_bootstrap.cc:492] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: Bootstrap starting.
I20260811 04:41:19.582437 24263 tablet_bootstrap.cc:654] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:19.583703 24264 tablet_bootstrap.cc:654] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:19.583753 24263 log.cc:824] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:19.584503 24265 tablet_bootstrap.cc:654] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:19.585052 24264 log.cc:824] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:19.585337 24265 log.cc:824] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:19.586162 24263 tablet_bootstrap.cc:492] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: No bootstrap required, opened a new log
I20260811 04:41:19.586359 24263 ts_tablet_manager.cc:1397] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 04:41:19.587217 24264 tablet_bootstrap.cc:492] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: No bootstrap required, opened a new log
I20260811 04:41:19.587469 24265 tablet_bootstrap.cc:492] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: No bootstrap required, opened a new log
I20260811 04:41:19.587522 24264 ts_tablet_manager.cc:1397] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 04:41:19.587709 24265 ts_tablet_manager.cc:1397] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:41:19.590204 24263 raft_consensus.cc:348] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.590596 24263 raft_consensus.cc:374] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:19.590715 24263 raft_consensus.cc:729] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b61460988d49e9b2ab65b6e986fe8d, State: Initialized, Role: FOLLOWER
I20260811 04:41:19.591215 24263 consensus_queue.cc:260] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.591969 24097 heartbeater.cc:503] Master 127.22.144.190:44347 was elected leader, sending a full tablet report...
I20260811 04:41:19.592351 24263 ts_tablet_manager.cc:1428] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:41:19.592806 24264 raft_consensus.cc:348] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.593175 24264 raft_consensus.cc:374] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:19.593291 24264 raft_consensus.cc:729] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7dd31310c10400780cf7ad643ead295, State: Initialized, Role: FOLLOWER
I20260811 04:41:19.593669 24265 raft_consensus.cc:348] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.593868 24264 consensus_queue.cc:260] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.593956 24265 raft_consensus.cc:374] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:19.594025 24265 raft_consensus.cc:729] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7267a5ca183941b2a0272d40c3a10153, State: Initialized, Role: FOLLOWER
I20260811 04:41:19.594506 24265 consensus_queue.cc:260] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.594775 24244 heartbeater.cc:503] Master 127.22.144.190:44347 was elected leader, sending a full tablet report...
I20260811 04:41:19.595367 23950 heartbeater.cc:503] Master 127.22.144.190:44347 was elected leader, sending a full tablet report...
I20260811 04:41:19.595726 24265 ts_tablet_manager.cc:1428] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260811 04:41:19.597734 24264 ts_tablet_manager.cc:1428] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 04:41:19.651892 24270 raft_consensus.cc:482] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:19.652210 24270 raft_consensus.cc:504] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.653653 24270 leader_election.cc:288] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7267a5ca183941b2a0272d40c3a10153 (127.22.144.129:39705), a7dd31310c10400780cf7ad643ead295 (127.22.144.131:35927)
I20260811 04:41:19.659240 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.654124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:38227 (local address 127.22.144.129:39705)
0811 04:41:19.654375 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.654380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.654402 (+ 22us) server_negotiation.cc:407] Connection header received
0811 04:41:19.654464 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.654466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.654538 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.654634 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:19.655018 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.655997 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.657838 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.658456 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.658605 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.658674 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.659001 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.659077 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260811 04:41:19.659250 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.653898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39705 (local address 127.22.144.130:38227)
0811 04:41:19.654055 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.654074 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.654185 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.654692 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.654695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.654719 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:19.654875 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.654884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.656125 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.656128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.657688 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.657701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.658612 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.658619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.658836 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.658864 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.658992 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.659089 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 04:41:19.659337 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.654245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:35927 (local address 127.22.144.130:41645)
0811 04:41:19.654705 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:19.654719 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:19.654820 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:19.655252 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:19.655255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:19.655276 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:19.655423 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.655429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.656413 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.656416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.657757 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:19.657768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.658830 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.658834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:19.659009 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.659033 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:41:19.659110 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 04:41:19.659170 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":155,"threads_started":1}
I20260811 04:41:19.659358 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.654451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:41645 (local address 127.22.144.131:35927)
0811 04:41:19.654641 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.654646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.654817 (+ 171us) server_negotiation.cc:407] Connection header received
0811 04:41:19.654918 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.654920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.655044 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.655130 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:19.655557 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.656303 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.657956 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.658547 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.658678 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.658737 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.659152 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.659229 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:41:19.660259 24193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3310b09e22204cecb6deb181270e01b4" candidate_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7dd31310c10400780cf7ad643ead295" is_pre_election: true
I20260811 04:41:19.660416 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3310b09e22204cecb6deb181270e01b4" candidate_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7267a5ca183941b2a0272d40c3a10153" is_pre_election: true
I20260811 04:41:19.660709 23897 raft_consensus.cc:2454] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b61460988d49e9b2ab65b6e986fe8d in term 0.
I20260811 04:41:19.660710 24193 raft_consensus.cc:2454] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b61460988d49e9b2ab65b6e986fe8d in term 0.
I20260811 04:41:19.661350 23977 leader_election.cc:302] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7dd31310c10400780cf7ad643ead295, c1b61460988d49e9b2ab65b6e986fe8d; no voters:
I20260811 04:41:19.661690 24270 raft_consensus.cc:2781] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:19.661793 24270 raft_consensus.cc:482] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:19.661846 24270 raft_consensus.cc:3037] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:19.663138 24270 raft_consensus.cc:504] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.663555 24270 leader_election.cc:288] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [CANDIDATE]: Term 1 election: Requested vote from peers 7267a5ca183941b2a0272d40c3a10153 (127.22.144.129:39705), a7dd31310c10400780cf7ad643ead295 (127.22.144.131:35927)
I20260811 04:41:19.664073 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3310b09e22204cecb6deb181270e01b4" candidate_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7267a5ca183941b2a0272d40c3a10153"
I20260811 04:41:19.664073 24193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3310b09e22204cecb6deb181270e01b4" candidate_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7dd31310c10400780cf7ad643ead295"
I20260811 04:41:19.664216 24193 raft_consensus.cc:3037] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:19.664216 23897 raft_consensus.cc:3037] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:19.666144 23897 raft_consensus.cc:2454] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b61460988d49e9b2ab65b6e986fe8d in term 1.
I20260811 04:41:19.666370 24193 raft_consensus.cc:2454] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b61460988d49e9b2ab65b6e986fe8d in term 1.
I20260811 04:41:19.666504 23977 leader_election.cc:302] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7267a5ca183941b2a0272d40c3a10153, c1b61460988d49e9b2ab65b6e986fe8d; no voters:
I20260811 04:41:19.666724 24270 raft_consensus.cc:2781] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:19.666997 24270 raft_consensus.cc:686] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [term 1 LEADER]: Becoming Leader. State: Replica: c1b61460988d49e9b2ab65b6e986fe8d, State: Running, Role: LEADER
I20260811 04:41:19.667348 24270 consensus_queue.cc:237] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } }
I20260811 04:41:19.670782 23727 catalog_manager.cc:5495] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b61460988d49e9b2ab65b6e986fe8d (127.22.144.130). New cstate: current_term: 1 leader_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1b61460988d49e9b2ab65b6e986fe8d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 33949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7267a5ca183941b2a0272d40c3a10153" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7dd31310c10400780cf7ad643ead295" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 35927 } health_report { overall_health: UNKNOWN } } }
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:19.700565 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.684301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48240 (local address 127.22.144.129:39705)
0811 04:41:19.684492 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.684497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.684734 (+ 237us) server_negotiation.cc:407] Connection header received
0811 04:41:19.684915 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.684917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.684972 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.685065 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.685887 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.686414 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.687393 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.687621 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.687734 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.695837 (+ 8103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.695854 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.695870 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.695908 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.698447 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.699092 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.699098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.699100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.699197 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.699514 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.699523 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.699524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.699817 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.700055 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.700259 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.700393 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:19.732793 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.722246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.22.144.129:39705)
0811 04:41:19.722484 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.722489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.722631 (+ 142us) server_negotiation.cc:407] Connection header received
0811 04:41:19.722839 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.722842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.722890 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.722994 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.723849 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.724422 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.725366 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.725732 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.725853 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.728735 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.728738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.728741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.728770 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.730596 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.731253 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.731260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.731262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.731360 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.731702 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.731706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.731708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.731961 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.732124 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.732467 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.732618 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260811 04:41:19.753564 24245 tablet.cc:2369] T 3310b09e22204cecb6deb181270e01b4 P a7dd31310c10400780cf7ad643ead295: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[23677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
W20260811 04:41:19.762787 24098 tablet.cc:2369] T 3310b09e22204cecb6deb181270e01b4 P c1b61460988d49e9b2ab65b6e986fe8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:19.766814 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:19.755953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.22.144.129:39705)
0811 04:41:19.756148 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:19.756154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:19.756330 (+ 176us) server_negotiation.cc:407] Connection header received
0811 04:41:19.756532 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:19.756535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:19.756609 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:19.756723 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:19.757549 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.758119 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.759132 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:19.759392 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:19.759520 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:19.762643 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:19.762646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:19.762648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:19.762678 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:19.764383 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.765002 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.765004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.765006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.765096 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:19.765545 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:19.765550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:19.765557 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:19.765808 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:19.765933 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:19.766359 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 04:41:19.766603 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 04:41:19.768424 23851 tablet_service.cc:2908] Rejecting scan request for tablet 3310b09e22204cecb6deb181270e01b4: Uninitialized: clean time has not yet been initialized
W20260811 04:41:19.771523 23951 tablet.cc:2369] T 3310b09e22204cecb6deb181270e01b4 P 7267a5ca183941b2a0272d40c3a10153: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:19.771950 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23802
I20260811 04:41:19.780958 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23956
I20260811 04:41:19.791453 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 24102
I20260811 04:41:19.800153 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 23690
2026-08-11T04:41:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2626 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[24290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[24290](info): set up 2 sockets
Aug 11 04:41:19 dist-test-slave-ss4n krb5kdc[24290](info): commencing operation
krb5kdc: starting...
W20260811 04:41:20.953445 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:21.097808 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42369
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:21.224372 24304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:21.224723 24304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:21.224797 24304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:21.228997 24304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:21.229099 24304 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 04:41:21.229131 24304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:21.229202 24304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:21.229256 24304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:21.229298 24304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:21.234925 24304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42369
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:21.236133 24304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:21.237215 24304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:21.246490 24312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:21.246482 24310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:21.246482 24309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:21.246605 24304 server_base.cc:1034] running on GCE node
I20260811 04:41:21.247488 24304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:21.248761 24304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:21.249971 24304 hybrid_clock.cc:648] HybridClock initialized: now 1786423281249947 us; error 41 us; skew 500 ppm
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:21.253127 24304 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:21.254196 24304 webserver.cc:460] Webserver started at http://127.22.144.190:37123/ using document root <none> and password file <none>
I20260811 04:41:21.254783 24304 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:21.254878 24304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:21.255121 24304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:21.257244 24304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "0af5159d89494d35b5b31510a0b9a284"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "0aaa07eba1d7cabe167ec52e556aaf31"
server_key_iv: "4fd3e273410e2847f6baf3e0a9c11141"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.257881 24304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "0af5159d89494d35b5b31510a0b9a284"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "0aaa07eba1d7cabe167ec52e556aaf31"
server_key_iv: "4fd3e273410e2847f6baf3e0a9c11141"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.261837 24304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:21.264446 24320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:21.265913 24304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:21.266063 24304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "0af5159d89494d35b5b31510a0b9a284"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "0aaa07eba1d7cabe167ec52e556aaf31"
server_key_iv: "4fd3e273410e2847f6baf3e0a9c11141"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.266189 24304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:21.285436 24304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:21.289192 24304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:21.289417 24304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:21.298655 24304 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:42369
I20260811 04:41:21.298691 24388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:42369 every 8 connection(s)
I20260811 04:41:21.299674 24304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:21.303071 24389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:21.306916 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 24304
I20260811 04:41:21.307070 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:21.307376 23106 external_mini_cluster.cc:1428] Setting key 20802dc18bfde0943c54ef047f40851b
I20260811 04:41:21.310200 24389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: Bootstrap starting.
I20260811 04:41:21.312713 24389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:21.313620 24389 log.cc:824] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:21.316007 24389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: No bootstrap required, opened a new log
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:21.320586 24389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af5159d89494d35b5b31510a0b9a284" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42369 } }
I20260811 04:41:21.320889 24389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:21.320991 24389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af5159d89494d35b5b31510a0b9a284, State: Initialized, Role: FOLLOWER
I20260811 04:41:21.321663 24389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af5159d89494d35b5b31510a0b9a284" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42369 } }
I20260811 04:41:21.321817 24389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:21.321916 24389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:21.322042 24389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:21.323141 24389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af5159d89494d35b5b31510a0b9a284" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42369 } }
I20260811 04:41:21.323624 24389 leader_election.cc:302] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af5159d89494d35b5b31510a0b9a284; no voters:
I20260811 04:41:21.324065 24389 leader_election.cc:288] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:21.324188 24395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:21.324437 24395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [term 1 LEADER]: Becoming Leader. State: Replica: 0af5159d89494d35b5b31510a0b9a284, State: Running, Role: LEADER
I20260811 04:41:21.324810 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.309087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.22.144.190:42369)
0811 04:41:21.309711 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:21.309729 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:21.309771 (+ 42us) server_negotiation.cc:407] Connection header received
0811 04:41:21.310587 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:21.310610 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:21.310986 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:21.311364 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:21.312305 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.313554 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.314812 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.315344 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.315535 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.318742 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:21.318748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:21.318765 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:21.318798 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:21.320971 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.321669 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.321676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.321681 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.321868 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.322244 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.322249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.322252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.322725 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:21.322969 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:21.323339 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 04:41:21.323676 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":167,"threads_started":1}
I20260811 04:41:21.324843 24395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af5159d89494d35b5b31510a0b9a284" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42369 } }
I20260811 04:41:21.325646 24389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:21.326226 24395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af5159d89494d35b5b31510a0b9a284. Latest consensus state: current_term: 1 leader_uuid: "0af5159d89494d35b5b31510a0b9a284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af5159d89494d35b5b31510a0b9a284" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42369 } } }
I20260811 04:41:21.326359 24395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:21.327883 24396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af5159d89494d35b5b31510a0b9a284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af5159d89494d35b5b31510a0b9a284" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42369 } } }
I20260811 04:41:21.328070 24396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:21.328029 24399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:21.331822 24399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:21.338444 24399 catalog_manager.cc:1347] Generated new cluster ID: 48d3257fe98949cfa4e0b12f146aee50
I20260811 04:41:21.338538 24399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:21.357630 24399 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:21.358968 24399 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:21.365551 24399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: Generated new TSK 0
I20260811 04:41:21.366385 24399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:21.411309 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42369
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:21.534169 24417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:21.534484 24417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:21.534701 24417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:21.538718 24417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:21.538808 24417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:21.538883 24417 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:21.538966 24417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:21.544684 24417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:42369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:21.545905 24417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:21.546967 24417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:21.557286 24425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:21.557224 24422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:21.557263 24417 server_base.cc:1034] running on GCE node
W20260811 04:41:21.557276 24423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:21.558008 24417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:21.558706 24417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:21.559900 24417 hybrid_clock.cc:648] HybridClock initialized: now 1786423281559881 us; error 36 us; skew 500 ppm
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:21.562877 24417 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:21.563788 24417 webserver.cc:460] Webserver started at http://127.22.144.129:38413/ using document root <none> and password file <none>
I20260811 04:41:21.564373 24417 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:21.564473 24417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:21.564792 24417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:21.567004 24417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "e053154c0fe64a8e998b8c5930edeeb8"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "326592a4948232799120fe2b13152007"
server_key_iv: "05b5f8ca42673e646fe510b3425fa222"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.567646 24417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "e053154c0fe64a8e998b8c5930edeeb8"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "326592a4948232799120fe2b13152007"
server_key_iv: "05b5f8ca42673e646fe510b3425fa222"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.571926 24417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:21.574533 24432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:21.576010 24417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:21.576210 24417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "e053154c0fe64a8e998b8c5930edeeb8"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "326592a4948232799120fe2b13152007"
server_key_iv: "05b5f8ca42673e646fe510b3425fa222"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.576347 24417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:21.590240 24417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:21.593330 24417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:21.593621 24417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:21.595243 24417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:21.595325 24417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:21.595394 24417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:21.595444 24417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:21.606422 24417 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:38117
I20260811 04:41:21.607333 24417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:21.607681 24555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:38117 every 8 connection(s)
I20260811 04:41:21.610109 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 24417
I20260811 04:41:21.610229 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:21.610641 23106 external_mini_cluster.cc:1428] Setting key 184fb88ebea81853bb0ad401393f0a2d
I20260811 04:41:21.611200 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.596138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:50501 (local address 127.22.144.190:42369)
0811 04:41:21.596266 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:21.596270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:21.597342 (+ 1072us) server_negotiation.cc:407] Connection header received
0811 04:41:21.598421 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:21.598425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:21.598490 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:21.598634 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:21.600415 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.601327 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.603357 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.603606 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.603696 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.607018 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:21.607022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:21.607025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:21.607060 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:21.608802 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.609395 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.609399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.609401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.609539 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.609908 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.609913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.609916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.610202 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:21.610315 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:21.610918 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 04:41:21.611071 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 04:41:21.612085 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.596681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.22.144.129:50501)
0811 04:41:21.597187 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:21.597235 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:21.598168 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:21.599083 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:21.599094 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:21.599535 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:21.600239 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.600263 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.601515 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.601523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.603209 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.603220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.603784 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.603798 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.604025 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.604816 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:21.604844 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:21.606818 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:21.608946 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.608955 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.608971 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.609288 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.609645 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.609648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.609649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.609789 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.610326 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:21.610334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:21.610595 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 04:41:21.610801 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 04:41:21.611137 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":158,"threads_started":1}
I20260811 04:41:21.613794 24556 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42369
I20260811 04:41:21.613997 24556 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:21.614681 24556 heartbeater.cc:511] Master 127.22.144.190:42369 requested a full tablet report, sending...
I20260811 04:41:21.616904 24337 ts_manager.cc:194] Registered new tserver with Master: e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117)
I20260811 04:41:21.618474 24337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:50501
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:21.626168 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.616462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.22.144.129:47475)
0811 04:41:21.616769 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:21.616784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:21.616877 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:21.617899 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:21.617902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:21.618151 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:21.618270 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.618277 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.619349 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.619355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.620374 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.620391 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.620995 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.621001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.621137 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.621805 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:21.621821 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:21.622381 (+ 560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:21.624293 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.624300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.624305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.624569 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.624901 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.624904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.624906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.625021 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.625532 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:21.625539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:21.625670 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 04:41:21.625805 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 04:41:21.625990 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260811 04:41:21.626151 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.616326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:47475 (local address 127.22.144.190:42369)
0811 04:41:21.617681 (+ 1355us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:21.617684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:21.617697 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:21.617732 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:21.617734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:21.617773 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:21.617843 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:21.618412 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.619200 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.620525 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.620849 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.620938 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.622577 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:21.622580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:21.622582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:21.622651 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:21.624170 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.624710 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.624713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.624715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.624796 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.625141 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.625144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.625146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.625389 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:21.625470 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:21.625857 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 04:41:21.626025 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:21.648813 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42369
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:21.774006 24566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:21.774384 24566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:21.774478 24566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:21.778908 24566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:21.778999 24566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:21.779070 24566 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:21.779153 24566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:21.784885 24566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:42369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:21.786140 24566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:21.787277 24566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:21.797655 24571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:21.797655 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:21.797670 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:21.798020 24566 server_base.cc:1034] running on GCE node
I20260811 04:41:21.798571 24566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:21.799220 24566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:21.800417 24566 hybrid_clock.cc:648] HybridClock initialized: now 1786423281800396 us; error 40 us; skew 500 ppm
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:21.803519 24566 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:21.804375 24566 webserver.cc:460] Webserver started at http://127.22.144.130:33169/ using document root <none> and password file <none>
I20260811 04:41:21.804913 24566 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:21.804975 24566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:21.805140 24566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:21.807260 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "0d244af8860e40debe852d5b48d17141"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "3c82ea977b3ecc8ec5490c2a188d669a"
server_key_iv: "269f6e17508c4d67893bbbf544b9aa06"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.807785 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "0d244af8860e40debe852d5b48d17141"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "3c82ea977b3ecc8ec5490c2a188d669a"
server_key_iv: "269f6e17508c4d67893bbbf544b9aa06"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.811581 24566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 04:41:21.814277 24582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:21.815604 24566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 04:41:21.815709 24566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "0d244af8860e40debe852d5b48d17141"
format_stamp: "Formatted at 2026-08-11 04:41:21 on dist-test-slave-ss4n"
server_key: "3c82ea977b3ecc8ec5490c2a188d669a"
server_key_iv: "269f6e17508c4d67893bbbf544b9aa06"
server_key_version: "encryptionkey@0"
I20260811 04:41:21.815805 24566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:21.825704 24566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:21.829310 24566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:21.829470 24566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:21.830921 24566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:21.830969 24566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:21.831009 24566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:21.831075 24566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:21 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:21.842832 24566 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:40393
I20260811 04:41:21.843700 24714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:40393 every 8 connection(s)
I20260811 04:41:21.844084 24566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:21.845850 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.831946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:45669 (local address 127.22.144.190:42369)
0811 04:41:21.832083 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:21.832087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:21.833149 (+ 1062us) server_negotiation.cc:407] Connection header received
0811 04:41:21.834433 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:21.834437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:21.834503 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:21.834562 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:21.836027 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.836944 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.838364 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.838727 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.838840 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.841663 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:21.841667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:21.841670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:21.841700 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:21.843454 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.844151 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.844155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.844157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.844257 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.844602 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.844605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.844607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.844812 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:21.844928 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:21.845534 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 04:41:21.845671 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:41:21.846586 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.832458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.22.144.130:45669)
0811 04:41:21.832989 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:21.833035 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:21.834130 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:21.834968 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:21.834975 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:21.835301 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:21.835867 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.835886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.837126 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.837129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.838230 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.838241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.838890 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.838894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.839082 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.839689 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:21.839705 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:21.841415 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:21.843580 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.843587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.843624 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.843997 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.844342 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.844344 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.844346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.844485 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.844926 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:21.844933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:21.845241 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 04:41:21.845431 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 04:41:21.845779 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":137,"threads_started":1}
I20260811 04:41:21.847165 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 24566
I20260811 04:41:21.847293 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:21.847592 23106 external_mini_cluster.cc:1428] Setting key 16a8c0bd5114e6a4ef63260032a74cb0
I20260811 04:41:21.847936 24715 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42369
I20260811 04:41:21.848083 24715 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:21.848687 24715 heartbeater.cc:511] Master 127.22.144.190:42369 requested a full tablet report, sending...
I20260811 04:41:21.849870 24338 ts_manager.cc:194] Registered new tserver with Master: 0d244af8860e40debe852d5b48d17141 (127.22.144.130:40393)
I20260811 04:41:21.850708 24338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:45669
I20260811 04:41:21.858819 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.850257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.22.144.130:40303)
0811 04:41:21.850526 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:21.850545 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:21.850615 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:21.851517 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:21.851528 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:21.851729 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:21.851832 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.851838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.852667 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.852671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.853320 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:21.853329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.854047 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.854054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:21.854159 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.854730 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:21.854741 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:21.855106 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:21.857101 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.857103 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.857105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.857306 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.857667 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:21.857669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:21.857671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:21.857755 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:21.858278 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:21.858281 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:21.858358 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 04:41:21.858438 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 04:41:21.858547 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":110,"threads_started":1}
I20260811 04:41:21.858881 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:21.851194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:40303 (local address 127.22.144.190:42369)
0811 04:41:21.851310 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:21.851314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:21.851326 (+ 12us) server_negotiation.cc:407] Connection header received
0811 04:41:21.851359 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:21.851362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:21.851404 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:21.851500 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:21.851953 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.852520 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.853469 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:21.853907 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:21.854047 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:21.855257 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:21.855264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:21.855270 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:21.855304 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:21.856979 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.857415 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.857420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.857424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.857562 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:21.857858 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:21.857864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:21.857868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:21.858174 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:21.858300 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:21.858557 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 04:41:21.858707 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:21.882247 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42369
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:22.001883 24725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:22.002151 24725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:22.002250 24725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:22.006557 24725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:22.006628 24725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:22.006669 24725 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:22.006702 24725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:22.012259 24725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:42369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:22.013387 24725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:22.014544 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:22.025777 24731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:22.025847 24733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:22.025794 24730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:22.025842 24725 server_base.cc:1034] running on GCE node
I20260811 04:41:22.026486 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:22.027320 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:22.028503 24725 hybrid_clock.cc:648] HybridClock initialized: now 1786423282028494 us; error 75 us; skew 500 ppm
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423282, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:22.031677 24725 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:22.032598 24725 webserver.cc:460] Webserver started at http://127.22.144.131:33709/ using document root <none> and password file <none>
I20260811 04:41:22.033180 24725 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:22.033244 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:22.033577 24725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:22.035772 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0"
format_stamp: "Formatted at 2026-08-11 04:41:22 on dist-test-slave-ss4n"
server_key: "8224bac0b70ce95a70a215edd6d99032"
server_key_iv: "48ec5288f26146659768f2f269bddc92"
server_key_version: "encryptionkey@0"
I20260811 04:41:22.036384 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0"
format_stamp: "Formatted at 2026-08-11 04:41:22 on dist-test-slave-ss4n"
server_key: "8224bac0b70ce95a70a215edd6d99032"
server_key_iv: "48ec5288f26146659768f2f269bddc92"
server_key_version: "encryptionkey@0"
I20260811 04:41:22.040730 24725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:22.043551 24740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:22.044796 24725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 04:41:22.044952 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0"
format_stamp: "Formatted at 2026-08-11 04:41:22 on dist-test-slave-ss4n"
server_key: "8224bac0b70ce95a70a215edd6d99032"
server_key_iv: "48ec5288f26146659768f2f269bddc92"
server_key_version: "encryptionkey@0"
I20260811 04:41:22.045077 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:22.065065 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:22.068802 24725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:22.069038 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:22.070909 24725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:22.070983 24725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:22.071056 24725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:22.071085 24725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423282, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:22.082635 24725 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:42743
I20260811 04:41:22.082715 24866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:42743 every 8 connection(s)
I20260811 04:41:22.083940 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:22.086616 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.071799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:48223 (local address 127.22.144.190:42369)
0811 04:41:22.071985 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.071993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.072929 (+ 936us) server_negotiation.cc:407] Connection header received
0811 04:41:22.074326 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.074330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.074394 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.074494 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:22.076341 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.077329 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.078849 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.079110 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.079231 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.082550 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:22.082556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:22.082560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:22.082595 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:22.084385 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.084982 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.084988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.084992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.085102 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.085544 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.085550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.085554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.085773 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:22.085859 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.086384 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.086492 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:41:22.087558 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.072238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.22.144.131:48223)
0811 04:41:22.072744 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.072798 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.073790 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.074719 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.074732 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.075279 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:22.076072 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.076111 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.077498 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.077505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.078664 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.078676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.079293 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.079297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.079496 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.080232 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:22.080262 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:22.082244 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:22.084499 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:22.084508 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:22.084524 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:22.084857 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:22.085232 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:22.085235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:22.085237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:22.085386 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:22.085880 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:22.085886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:22.086146 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.086312 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.086630 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":154,"threads_started":1}
I20260811 04:41:22.088840 24867 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42369
I20260811 04:41:22.088992 24867 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:22.089646 24867 heartbeater.cc:511] Master 127.22.144.190:42369 requested a full tablet report, sending...
I20260811 04:41:22.090692 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 24725
I20260811 04:41:22.090849 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:22.091140 23106 external_mini_cluster.cc:1428] Setting key a80e90ea9d26c3705a883fc7fcf3ba18
I20260811 04:41:22.091132 24337 ts_manager.cc:194] Registered new tserver with Master: 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743)
I20260811 04:41:22.091993 24337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:48223
I20260811 04:41:22.093597 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:41:22.100716 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.091630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.22.144.131:56697)
0811 04:41:22.091942 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.091958 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.092074 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.092542 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.092547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.092785 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:22.092893 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.092899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.093980 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.093983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.094737 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.094748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.095486 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.095492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.095588 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.096205 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:22.096226 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:22.096847 (+ 621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:22.098800 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:22.098807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:22.098813 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:22.099167 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:22.099489 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:22.099494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:22.099498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:22.099631 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:22.100042 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:22.100047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:22.100172 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.100323 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.100508 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":116,"threads_started":1}
I20260811 04:41:22.100714 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.091672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:56697 (local address 127.22.144.190:42369)
0811 04:41:22.091922 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.091926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.092052 (+ 126us) server_negotiation.cc:407] Connection header received
0811 04:41:22.092265 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.092269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.092322 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.092408 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:22.093030 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.093833 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.094875 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.095303 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.095497 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.097006 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:22.097010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:22.097012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:22.097037 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:22.098674 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.099272 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.099275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.099276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.099354 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.099743 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.099746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.099747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.099930 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:22.100015 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.100385 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.100539 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:22.125847 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.116111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.22.144.190:42369)
0811 04:41:22.116270 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.116274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.116289 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:22.116333 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.116336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.116376 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.116454 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:22.117328 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.117970 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.118920 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.119342 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.119448 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.122222 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:22.122226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:22.122228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:22.122260 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:22.123718 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.124207 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.124210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.124212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.124296 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.124639 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.124642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.124643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.124853 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:22.124966 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.125384 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.125647 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 04:41:22.129304 24337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:22.132525 24337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:22.148398 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.142014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.22.144.131:42743)
0811 04:41:22.143234 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.143246 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.143271 (+ 25us) server_negotiation.cc:407] Connection header received
0811 04:41:22.143348 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.143354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.143579 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.144114 (+ 535us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.144260 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.145445 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.146859 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.147670 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.147815 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.147954 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.148135 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.148236 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":118,"threads_started":1}
I20260811 04:41:22.148577 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.142722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.22.144.130:40393)
0811 04:41:22.143233 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.143247 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.143590 (+ 343us) server_negotiation.cc:407] Connection header received
0811 04:41:22.143856 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.143862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.144042 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.144198 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.144427 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.145177 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.147098 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.147947 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.148120 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.148241 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.148286 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.148361 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":130,"threads_started":1}
I20260811 04:41:22.148592 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.141846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:42743 (local address 127.0.0.1:54484)
0811 04:41:22.142405 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.142460 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.142760 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.143755 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.143763 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.143789 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.143949 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.143957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.145615 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.145619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.146732 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.146742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.147811 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.147814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.147900 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.147934 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.148090 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.148222 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260811 04:41:22.148829 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.142543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:40393 (local address 127.0.0.1:54400)
0811 04:41:22.143481 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.143503 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.143667 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.144182 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.144186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.144205 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.144301 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.144309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.145311 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.145314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.146960 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.146973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.148077 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.148083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.148164 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.148185 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.148244 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.148308 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":93,"thread_start_us":120,"threads_started":1}
I20260811 04:41:22.150300 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.141967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:38117 (local address 127.0.0.1:45692)
0811 04:41:22.142825 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.142844 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.143320 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.143979 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.143983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.144006 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.144115 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.144121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.145554 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.145558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.146960 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.146973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.149895 (+ 2922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.149902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.150009 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.150033 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.150104 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.150176 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":134,"thread_start_us":104,"threads_started":1}
I20260811 04:41:22.150355 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.143018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.22.144.129:38117)
0811 04:41:22.143423 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.143436 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.143479 (+ 43us) server_negotiation.cc:407] Connection header received
0811 04:41:22.143562 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.143569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.143837 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.143994 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.144246 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.145398 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.148766 (+ 3368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.149772 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.149892 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.150090 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.150141 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.150230 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":96,"threads_started":1}
I20260811 04:41:22.152062 24797 tablet_service.cc:1467] Processing CreateTablet for tablet bc0b0fed8a2744d88ea9c09ca5f4a979 (DEFAULT_TABLE table=test-table [id=9b860d447ad94ea8bd56718543309c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:22.152482 24641 tablet_service.cc:1467] Processing CreateTablet for tablet bc0b0fed8a2744d88ea9c09ca5f4a979 (DEFAULT_TABLE table=test-table [id=9b860d447ad94ea8bd56718543309c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:22.153070 24797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0b0fed8a2744d88ea9c09ca5f4a979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.152850 24487 tablet_service.cc:1467] Processing CreateTablet for tablet bc0b0fed8a2744d88ea9c09ca5f4a979 (DEFAULT_TABLE table=test-table [id=9b860d447ad94ea8bd56718543309c8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:22.153564 24641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0b0fed8a2744d88ea9c09ca5f4a979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.153863 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0b0fed8a2744d88ea9c09ca5f4a979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.160251 24891 tablet_bootstrap.cc:492] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Bootstrap starting.
I20260811 04:41:22.161834 24892 tablet_bootstrap.cc:492] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Bootstrap starting.
I20260811 04:41:22.162773 24891 tablet_bootstrap.cc:654] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.163703 24891 log.cc:824] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:22.164021 24892 tablet_bootstrap.cc:654] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.164510 24894 tablet_bootstrap.cc:492] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Bootstrap starting.
I20260811 04:41:22.164862 24892 log.cc:824] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:22.166115 24891 tablet_bootstrap.cc:492] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: No bootstrap required, opened a new log
I20260811 04:41:22.166352 24891 ts_tablet_manager.cc:1397] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 04:41:22.166646 24894 tablet_bootstrap.cc:654] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.167019 24892 tablet_bootstrap.cc:492] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: No bootstrap required, opened a new log
I20260811 04:41:22.167205 24892 ts_tablet_manager.cc:1397] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:41:22.167469 24894 log.cc:824] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:22.169869 24894 tablet_bootstrap.cc:492] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: No bootstrap required, opened a new log
I20260811 04:41:22.170125 24894 ts_tablet_manager.cc:1397] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 04:41:22.170364 24891 raft_consensus.cc:348] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.170653 24891 raft_consensus.cc:374] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.170727 24891 raft_consensus.cc:729] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d6e9a0ce7e945aaa4dc170b9ff95ed0, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.171587 24891 consensus_queue.cc:260] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.171959 24892 raft_consensus.cc:348] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.172230 24892 raft_consensus.cc:374] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.172286 24892 raft_consensus.cc:729] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e053154c0fe64a8e998b8c5930edeeb8, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.172402 24867 heartbeater.cc:503] Master 127.22.144.190:42369 was elected leader, sending a full tablet report...
I20260811 04:41:22.172746 24891 ts_tablet_manager.cc:1428] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:41:22.172749 24892 consensus_queue.cc:260] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.173682 24892 ts_tablet_manager.cc:1428] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 04:41:22.174053 24556 heartbeater.cc:503] Master 127.22.144.190:42369 was elected leader, sending a full tablet report...
I20260811 04:41:22.174075 24894 raft_consensus.cc:348] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.174247 24894 raft_consensus.cc:374] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.174275 24894 raft_consensus.cc:729] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d244af8860e40debe852d5b48d17141, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.174642 24894 consensus_queue.cc:260] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.175202 24715 heartbeater.cc:503] Master 127.22.144.190:42369 was elected leader, sending a full tablet report...
I20260811 04:41:22.176483 24894 ts_tablet_manager.cc:1428] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260811 04:41:22.271039 24898 raft_consensus.cc:482] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:22.271306 24898 raft_consensus.cc:504] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.272255 24901 raft_consensus.cc:482] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:22.272442 24901 raft_consensus.cc:504] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.272907 24898 leader_election.cc:288] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117), 0d244af8860e40debe852d5b48d17141 (127.22.144.130:40393)
I20260811 04:41:22.274886 24901 leader_election.cc:288] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743), e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117)
I20260811 04:41:22.278743 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.273766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:42743 (local address 127.22.144.130:37101)
0811 04:41:22.274032 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.274046 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.274170 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.274583 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.274586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.274625 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.274735 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.274743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.275824 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.275827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.277184 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.277197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.278135 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.278139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.278368 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.278410 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.278483 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.278574 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":80}
I20260811 04:41:22.279070 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.273451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:42963 (local address 127.22.144.129:38117)
0811 04:41:22.273645 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.273652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.274128 (+ 476us) server_negotiation.cc:407] Connection header received
0811 04:41:22.274342 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.274348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.274404 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.274463 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.275407 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.276183 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.277762 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.278359 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.278491 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.278534 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.278846 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.278926 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":94}
I20260811 04:41:22.279213 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.273813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:37101 (local address 127.22.144.131:42743)
0811 04:41:22.273921 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.273926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.274265 (+ 339us) server_negotiation.cc:407] Connection header received
0811 04:41:22.274319 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.274324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.274380 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.274454 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.274875 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.275695 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.277341 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.277973 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.278102 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.278155 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.279024 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.279098 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 04:41:22.279419 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.273249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:38117 (local address 127.22.144.131:42963)
0811 04:41:22.274035 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.274050 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.274201 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.274602 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.274605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.274625 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.275254 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.275265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.276325 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.276328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.277619 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.277630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.278546 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.278553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.278696 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.278724 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.279204 (+ 480us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.279281 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":289,"thread_start_us":149,"threads_started":1}
I20260811 04:41:22.279596 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.273249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:40393 (local address 127.22.144.131:42799)
0811 04:41:22.273623 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.273643 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.273849 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.274442 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.274446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.274525 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.274661 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.274668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.276026 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.276030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.277424 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.277435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.279141 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.279145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.279305 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.279328 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.279412 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.279490 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"spinlock_wait_cycles":1408}
I20260811 04:41:22.279903 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" is_pre_election: true
I20260811 04:41:22.280115 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.273364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:42799 (local address 127.22.144.130:40393)
0811 04:41:22.273524 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.273530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.274023 (+ 493us) server_negotiation.cc:407] Connection header received
0811 04:41:22.274091 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.274098 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.274160 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.274237 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.274810 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.275772 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.277673 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.278756 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.278867 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.279882 (+ 1015us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.279919 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.280001 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1261,"server-negotiator.queue_time_us":52}
I20260811 04:41:22.280126 24819 raft_consensus.cc:2454] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 0.
I20260811 04:41:22.280602 24663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d244af8860e40debe852d5b48d17141" is_pre_election: true
I20260811 04:41:22.280648 24583 leader_election.cc:302] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, 2d6e9a0ce7e945aaa4dc170b9ff95ed0; no voters:
I20260811 04:41:22.280791 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8" is_pre_election: true
I20260811 04:41:22.280836 24663 raft_consensus.cc:2454] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d6e9a0ce7e945aaa4dc170b9ff95ed0 in term 0.
I20260811 04:41:22.280961 24901 raft_consensus.cc:2781] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:22.281038 24901 raft_consensus.cc:482] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:22.281095 24901 raft_consensus.cc:3037] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.281085 24507 raft_consensus.cc:2454] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d6e9a0ce7e945aaa4dc170b9ff95ed0 in term 0.
I20260811 04:41:22.281325 24743 leader_election.cc:302] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, 2d6e9a0ce7e945aaa4dc170b9ff95ed0; no voters:
I20260811 04:41:22.281452 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.274679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:57401 (local address 127.22.144.129:38117)
0811 04:41:22.275687 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.275695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.275709 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:22.275754 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.275759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.275817 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.276653 (+ 836us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:22.276768 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.278433 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.279972 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.280602 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.281138 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.281199 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.281234 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.281297 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":942,"thread_start_us":103,"threads_started":1}
I20260811 04:41:22.281649 24898 raft_consensus.cc:2781] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:22.281729 24898 raft_consensus.cc:482] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:22.281775 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.274268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:38117 (local address 127.22.144.130:57401)
0811 04:41:22.275166 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.275179 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.275395 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.275929 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.275932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.275949 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:22.276536 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.276546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.278655 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.278660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.279840 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.279851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.280749 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.280753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.280870 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.280908 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.280963 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.281029 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260811 04:41:22.281780 24898 raft_consensus.cc:3037] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.282256 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8" is_pre_election: true
I20260811 04:41:22.282379 24507 raft_consensus.cc:2454] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 0.
I20260811 04:41:22.282701 24901 raft_consensus.cc:504] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.283118 24901 leader_election.cc:288] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 election: Requested vote from peers 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743), e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117)
I20260811 04:41:22.283361 24898 raft_consensus.cc:504] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.283533 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0"
I20260811 04:41:22.283509 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8"
I20260811 04:41:22.283645 24507 raft_consensus.cc:3037] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.283685 24819 raft_consensus.cc:2379] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0d244af8860e40debe852d5b48d17141 in current term 1: Already voted for candidate 2d6e9a0ce7e945aaa4dc170b9ff95ed0 in this term.
I20260811 04:41:22.284113 24506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8"
I20260811 04:41:22.284123 24663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0b0fed8a2744d88ea9c09ca5f4a979" candidate_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d244af8860e40debe852d5b48d17141"
I20260811 04:41:22.284252 24663 raft_consensus.cc:2379] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d6e9a0ce7e945aaa4dc170b9ff95ed0 in current term 1: Already voted for candidate 0d244af8860e40debe852d5b48d17141 in this term.
I20260811 04:41:22.284503 24898 leader_election.cc:288] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [CANDIDATE]: Term 1 election: Requested vote from peers e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117), 0d244af8860e40debe852d5b48d17141 (127.22.144.130:40393)
I20260811 04:41:22.284785 24742 leader_election.cc:302] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d6e9a0ce7e945aaa4dc170b9ff95ed0; no voters: 0d244af8860e40debe852d5b48d17141, e053154c0fe64a8e998b8c5930edeeb8
I20260811 04:41:22.284948 24898 raft_consensus.cc:2726] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 04:41:22.284984 24507 raft_consensus.cc:2454] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 1.
I20260811 04:41:22.285284 24583 leader_election.cc:302] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, e053154c0fe64a8e998b8c5930edeeb8; no voters: 2d6e9a0ce7e945aaa4dc170b9ff95ed0
I20260811 04:41:22.285434 24901 raft_consensus.cc:2781] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:22.285684 24901 raft_consensus.cc:686] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 1 LEADER]: Becoming Leader. State: Replica: 0d244af8860e40debe852d5b48d17141, State: Running, Role: LEADER
I20260811 04:41:22.286018 24901 consensus_queue.cc:237] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } }
I20260811 04:41:22.289850 24338 catalog_manager.cc:5495] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d244af8860e40debe852d5b48d17141 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:22.311537 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.306774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54406 (local address 127.22.144.130:40393)
0811 04:41:22.306909 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.306914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.306932 (+ 18us) server_negotiation.cc:407] Connection header received
0811 04:41:22.307121 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.307125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.307197 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.307320 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:22.307593 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.308478 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.309558 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.309942 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.310053 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.310428 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:22.311017 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:22.311143 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.311271 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.311341 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:41:22.318063 24507 raft_consensus.cc:1261] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Refusing update from remote peer 0d244af8860e40debe852d5b48d17141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:22.318085 24819 raft_consensus.cc:1261] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Refusing update from remote peer 0d244af8860e40debe852d5b48d17141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:22.318744 24901 consensus_queue.cc:1035] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:22.319005 24905 consensus_queue.cc:1035] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:22.327150 24908 mvcc.cc:205] Tried to move back new op lower bound from 7317189764367810560 to 7317189764246183936. Current Snapshot: MvccSnapshot[applied={T|T < 7317189764367810560}]
I20260811 04:41:22.329099 24910 mvcc.cc:205] Tried to move back new op lower bound from 7317189764367810560 to 7317189764246183936. Current Snapshot: MvccSnapshot[applied={T|T < 7317189764367810560}]
I20260811 04:41:22.342644 24335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:22.342901 24335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:22.346230 24335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: Sending DeleteTablet for 3 replicas of tablet bc0b0fed8a2744d88ea9c09ca5f4a979
I20260811 04:41:22.347354 24487 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0b0fed8a2744d88ea9c09ca5f4a979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:45692
I20260811 04:41:22.347523 24641 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0b0fed8a2744d88ea9c09ca5f4a979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54400
I20260811 04:41:22.348094 24797 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0b0fed8a2744d88ea9c09ca5f4a979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54484
I20260811 04:41:22.348402 24921 tablet_replica.cc:330] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: stopping tablet replica
I20260811 04:41:22.348627 24922 tablet_replica.cc:330] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: stopping tablet replica
I20260811 04:41:22.348623 24335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 04:41:22.348996 24922 raft_consensus.cc:2229] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 04:41:22.349148 24335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:22.349208 24921 raft_consensus.cc:2229] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:22.349448 24922 raft_consensus.cc:2258] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:22.349946 24921 raft_consensus.cc:2258] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:22.350878 24919 tablet_replica.cc:330] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: stopping tablet replica
I20260811 04:41:22.351212 24919 raft_consensus.cc:2229] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:22.351223 24922 ts_tablet_manager.cc:1905] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:22.351541 24919 raft_consensus.cc:2258] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:22.354382 24922 ts_tablet_manager.cc:1918] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:22.354490 24922 log.cc:1196] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/bc0b0fed8a2744d88ea9c09ca5f4a979
I20260811 04:41:22.354904 24922 ts_tablet_manager.cc:1939] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Deleting consensus metadata
I20260811 04:41:22.355078 24921 ts_tablet_manager.cc:1905] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:22.356441 24321 catalog_manager.cc:4841] TS 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743): tablet bc0b0fed8a2744d88ea9c09ca5f4a979 (table test-table [id=9b860d447ad94ea8bd56718543309c8f]) successfully deleted
I20260811 04:41:22.356534 24797 tablet_service.cc:1467] Processing CreateTablet for tablet 633e1fd8a2284b4f852a6300c48b9b0e (DEFAULT_TABLE table=test-table [id=5e7f97977e9340cbb61b7b29dd95d476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:22.356791 24797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633e1fd8a2284b4f852a6300c48b9b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.357851 24487 tablet_service.cc:1467] Processing CreateTablet for tablet 633e1fd8a2284b4f852a6300c48b9b0e (DEFAULT_TABLE table=test-table [id=5e7f97977e9340cbb61b7b29dd95d476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:22.358194 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633e1fd8a2284b4f852a6300c48b9b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.358189 24921 ts_tablet_manager.cc:1918] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:22.358290 24921 log.cc:1196] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/bc0b0fed8a2744d88ea9c09ca5f4a979
I20260811 04:41:22.358683 24921 ts_tablet_manager.cc:1939] T bc0b0fed8a2744d88ea9c09ca5f4a979 P 0d244af8860e40debe852d5b48d17141: Deleting consensus metadata
I20260811 04:41:22.359056 24891 tablet_bootstrap.cc:492] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Bootstrap starting.
I20260811 04:41:22.360069 24891 tablet_bootstrap.cc:654] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.360352 24641 tablet_service.cc:1467] Processing CreateTablet for tablet 633e1fd8a2284b4f852a6300c48b9b0e (DEFAULT_TABLE table=test-table [id=5e7f97977e9340cbb61b7b29dd95d476]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:22.360644 24641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633e1fd8a2284b4f852a6300c48b9b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.360919 24323 catalog_manager.cc:4841] TS 0d244af8860e40debe852d5b48d17141 (127.22.144.130:40393): tablet bc0b0fed8a2744d88ea9c09ca5f4a979 (table test-table [id=9b860d447ad94ea8bd56718543309c8f]) successfully deleted
I20260811 04:41:22.361341 24892 tablet_bootstrap.cc:492] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Bootstrap starting.
I20260811 04:41:22.361666 24891 tablet_bootstrap.cc:492] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: No bootstrap required, opened a new log
I20260811 04:41:22.361740 24891 ts_tablet_manager.cc:1397] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:22.362355 24891 raft_consensus.cc:348] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.362470 24891 raft_consensus.cc:374] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.362501 24891 raft_consensus.cc:729] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d6e9a0ce7e945aaa4dc170b9ff95ed0, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.362641 24891 consensus_queue.cc:260] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.362757 24892 tablet_bootstrap.cc:654] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.362880 24891 ts_tablet_manager.cc:1428] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:22.363358 24894 tablet_bootstrap.cc:492] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Bootstrap starting.
I20260811 04:41:22.364574 24894 tablet_bootstrap.cc:654] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.364596 24919 ts_tablet_manager.cc:1905] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:22.366793 24894 tablet_bootstrap.cc:492] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: No bootstrap required, opened a new log
I20260811 04:41:22.366907 24894 ts_tablet_manager.cc:1397] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 04:41:22.367533 24892 tablet_bootstrap.cc:492] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: No bootstrap required, opened a new log
I20260811 04:41:22.367534 24894 raft_consensus.cc:348] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.367640 24892 ts_tablet_manager.cc:1397] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 04:41:22.367669 24894 raft_consensus.cc:374] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.367723 24894 raft_consensus.cc:729] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d244af8860e40debe852d5b48d17141, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.367882 24894 consensus_queue.cc:260] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.367945 24919 ts_tablet_manager.cc:1918] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:22.368089 24919 log.cc:1196] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/bc0b0fed8a2744d88ea9c09ca5f4a979
I20260811 04:41:22.368187 24894 ts_tablet_manager.cc:1428] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 04:41:22.368183 24892 raft_consensus.cc:348] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.368321 24892 raft_consensus.cc:374] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.368372 24892 raft_consensus.cc:729] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e053154c0fe64a8e998b8c5930edeeb8, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.368510 24892 consensus_queue.cc:260] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.368602 24919 ts_tablet_manager.cc:1939] T bc0b0fed8a2744d88ea9c09ca5f4a979 P e053154c0fe64a8e998b8c5930edeeb8: Deleting consensus metadata
I20260811 04:41:22.368811 24892 ts_tablet_manager.cc:1428] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:22.370690 24322 catalog_manager.cc:4841] TS e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117): tablet bc0b0fed8a2744d88ea9c09ca5f4a979 (table test-table [id=9b860d447ad94ea8bd56718543309c8f]) successfully deleted
I20260811 04:41:22.561764 24915 raft_consensus.cc:482] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:22.561928 24915 raft_consensus.cc:504] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.562475 24915 leader_election.cc:288] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117), 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743)
I20260811 04:41:22.563046 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633e1fd8a2284b4f852a6300c48b9b0e" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" is_pre_election: true
I20260811 04:41:22.563036 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633e1fd8a2284b4f852a6300c48b9b0e" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8" is_pre_election: true
I20260811 04:41:22.563185 24819 raft_consensus.cc:2454] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 0.
I20260811 04:41:22.563236 24507 raft_consensus.cc:2454] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 0.
I20260811 04:41:22.563547 24583 leader_election.cc:302] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, 2d6e9a0ce7e945aaa4dc170b9ff95ed0; no voters:
I20260811 04:41:22.563762 24915 raft_consensus.cc:2781] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:22.563836 24915 raft_consensus.cc:482] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:22.563906 24915 raft_consensus.cc:3037] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.565104 24915 raft_consensus.cc:504] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.565615 24915 leader_election.cc:288] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 election: Requested vote from peers e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117), 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743)
I20260811 04:41:22.566077 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633e1fd8a2284b4f852a6300c48b9b0e" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8"
I20260811 04:41:22.566120 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633e1fd8a2284b4f852a6300c48b9b0e" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0"
I20260811 04:41:22.566231 24507 raft_consensus.cc:3037] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.566273 24819 raft_consensus.cc:3037] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.567277 24507 raft_consensus.cc:2454] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 1.
I20260811 04:41:22.567487 24819 raft_consensus.cc:2454] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 1.
I20260811 04:41:22.567693 24584 leader_election.cc:302] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, e053154c0fe64a8e998b8c5930edeeb8; no voters:
I20260811 04:41:22.567919 24915 raft_consensus.cc:2781] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:22.568051 24915 raft_consensus.cc:686] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 1 LEADER]: Becoming Leader. State: Replica: 0d244af8860e40debe852d5b48d17141, State: Running, Role: LEADER
I20260811 04:41:22.568219 24915 consensus_queue.cc:237] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.570271 24338 catalog_manager.cc:5495] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d244af8860e40debe852d5b48d17141 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } health_report { overall_health: UNKNOWN } } }
W20260811 04:41:22.585770 24868 tablet.cc:2369] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:41:22.596647 24716 tablet.cc:2369] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:41:22.609373 24557 tablet.cc:2369] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423281, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:22.645117 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.635603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.22.144.190:42369)
0811 04:41:22.636055 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.636068 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.636166 (+ 98us) server_negotiation.cc:407] Connection header received
0811 04:41:22.636342 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.636345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.636394 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.636506 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:22.637254 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.637881 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.638695 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.638926 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.639047 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.641534 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:22.641537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:22.641539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:22.641560 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:22.643050 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.643630 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.643635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.643637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.643731 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.644063 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.644070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.644072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.644278 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:22.644399 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.644745 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.644958 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":373,"thread_start_us":133,"threads_started":1}
I20260811 04:41:22.645129 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.635543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42369 (local address 127.0.0.1:48500)
0811 04:41:22.636050 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.636069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.636238 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.636572 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.636576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.637014 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:22.637124 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.637131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.637992 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.637998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.638570 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.638578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.639045 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.639048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.639151 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.639740 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:22.639753 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:22.641347 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:22.643206 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:22.643220 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:22.643227 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:22.643493 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:22.643842 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:22.643845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:22.643847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:22.643951 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:22.644410 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:22.644430 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:22.644546 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.644681 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.644884 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":4,"thread_start_us":183,"threads_started":1}
I20260811 04:41:22.650249 24338 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:48500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 04:41:22.651145 24338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:22.656404 24487 tablet_service.cc:1467] Processing CreateTablet for tablet 40a7f857659f4e59924157adbd043169 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7905801edd84c1e8f5fc86ce7f379a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:22.656417 24641 tablet_service.cc:1467] Processing CreateTablet for tablet 40a7f857659f4e59924157adbd043169 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7905801edd84c1e8f5fc86ce7f379a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:22.656661 24797 tablet_service.cc:1467] Processing CreateTablet for tablet 40a7f857659f4e59924157adbd043169 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7905801edd84c1e8f5fc86ce7f379a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:22.656931 24641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a7f857659f4e59924157adbd043169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.656949 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a7f857659f4e59924157adbd043169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.657011 24797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a7f857659f4e59924157adbd043169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:22.659765 24894 tablet_bootstrap.cc:492] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: Bootstrap starting.
I20260811 04:41:22.660248 24892 tablet_bootstrap.cc:492] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: Bootstrap starting.
I20260811 04:41:22.660413 24891 tablet_bootstrap.cc:492] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Bootstrap starting.
I20260811 04:41:22.660751 24894 tablet_bootstrap.cc:654] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.661347 24892 tablet_bootstrap.cc:654] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.661535 24891 tablet_bootstrap.cc:654] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:22.662453 24894 tablet_bootstrap.cc:492] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: No bootstrap required, opened a new log
I20260811 04:41:22.662530 24894 ts_tablet_manager.cc:1397] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 04:41:22.662722 24892 tablet_bootstrap.cc:492] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: No bootstrap required, opened a new log
I20260811 04:41:22.662827 24892 ts_tablet_manager.cc:1397] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 04:41:22.662978 24894 raft_consensus.cc:348] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.663131 24894 raft_consensus.cc:374] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.663175 24894 raft_consensus.cc:729] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d244af8860e40debe852d5b48d17141, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.663319 24894 consensus_queue.cc:260] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.663303 24892 raft_consensus.cc:348] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.663443 24892 raft_consensus.cc:374] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.663486 24892 raft_consensus.cc:729] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e053154c0fe64a8e998b8c5930edeeb8, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.663614 24892 consensus_queue.cc:260] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.663640 24894 ts_tablet_manager.cc:1428] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 04:41:22.663647 24915 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.663861 24915 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.663861 24899 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.663983 24899 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.664619 24892 ts_tablet_manager.cc:1428] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 04:41:22.664981 24891 tablet_bootstrap.cc:492] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: No bootstrap required, opened a new log
I20260811 04:41:22.665078 24891 ts_tablet_manager.cc:1397] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 04:41:22.665591 24891 raft_consensus.cc:348] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.665724 24891 raft_consensus.cc:374] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:22.665788 24891 raft_consensus.cc:729] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d6e9a0ce7e945aaa4dc170b9ff95ed0, State: Initialized, Role: FOLLOWER
I20260811 04:41:22.665925 24891 consensus_queue.cc:260] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.666242 24891 ts_tablet_manager.cc:1428] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 04:41:22.666174 24898 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.666337 24898 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.739142 24915 raft_consensus.cc:482] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:22.739321 24915 raft_consensus.cc:504] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.739950 24915 leader_election.cc:288] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117), 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743)
I20260811 04:41:22.740531 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a7f857659f4e59924157adbd043169" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" is_pre_election: true
I20260811 04:41:22.740518 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a7f857659f4e59924157adbd043169" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8" is_pre_election: true
I20260811 04:41:22.740690 24819 raft_consensus.cc:2454] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 0.
I20260811 04:41:22.740690 24507 raft_consensus.cc:2454] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 0.
I20260811 04:41:22.741052 24583 leader_election.cc:302] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, 2d6e9a0ce7e945aaa4dc170b9ff95ed0; no voters:
I20260811 04:41:22.741289 24915 raft_consensus.cc:2781] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:22.741379 24915 raft_consensus.cc:482] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:22.741427 24915 raft_consensus.cc:3037] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.742776 24915 raft_consensus.cc:504] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.743129 24915 leader_election.cc:288] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 election: Requested vote from peers e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117), 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743)
I20260811 04:41:22.743607 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a7f857659f4e59924157adbd043169" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e053154c0fe64a8e998b8c5930edeeb8"
I20260811 04:41:22.743659 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a7f857659f4e59924157adbd043169" candidate_uuid: "0d244af8860e40debe852d5b48d17141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0"
I20260811 04:41:22.743773 24819 raft_consensus.cc:3037] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.743773 24507 raft_consensus.cc:3037] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:22.744969 24819 raft_consensus.cc:2454] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 1.
I20260811 04:41:22.744969 24507 raft_consensus.cc:2454] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d244af8860e40debe852d5b48d17141 in term 1.
I20260811 04:41:22.745324 24584 leader_election.cc:302] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d244af8860e40debe852d5b48d17141, e053154c0fe64a8e998b8c5930edeeb8; no voters:
I20260811 04:41:22.745570 24915 raft_consensus.cc:2781] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:22.745688 24915 raft_consensus.cc:686] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [term 1 LEADER]: Becoming Leader. State: Replica: 0d244af8860e40debe852d5b48d17141, State: Running, Role: LEADER
I20260811 04:41:22.745877 24915 consensus_queue.cc:237] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } }
I20260811 04:41:22.746601 24905 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: TxnStatusTablet state changed. Reason: New leader 0d244af8860e40debe852d5b48d17141. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.746730 24905 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:22.746991 24940 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 04:41:22.747316 24940 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 04:41:22.748122 24334 catalog_manager.cc:5495] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d244af8860e40debe852d5b48d17141 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } health_report { overall_health: UNKNOWN } } }
I20260811 04:41:22.773432 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.769032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:40393 (local address 127.0.0.1:54408)
0811 04:41:22.769209 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.769229 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.769359 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.769764 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.769767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.769776 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:22.769986 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.769993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.771176 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.771179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.771902 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.771913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.772619 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.772622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.772699 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.772757 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:22.773130 (+ 373us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:22.773150 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.773222 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.773308 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 04:41:22.773546 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.769178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54408 (local address 127.22.144.130:40393)
0811 04:41:22.769307 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.769312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.769329 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:41:22.769530 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.769535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.769599 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.769695 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:22.770117 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.771036 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.772081 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.772473 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.772592 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.772911 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:22.773012 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:22.773093 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.773260 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.773327 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 04:41:22.776571 24819 raft_consensus.cc:1261] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Refusing update from remote peer 0d244af8860e40debe852d5b48d17141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:22.776592 24507 raft_consensus.cc:1261] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Refusing update from remote peer 0d244af8860e40debe852d5b48d17141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:22.777057 24905 consensus_queue.cc:1035] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:22.777256 24915 consensus_queue.cc:1035] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:22.778901 24899 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: TxnStatusTablet state changed. Reason: New leader 0d244af8860e40debe852d5b48d17141. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.779023 24899 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.779120 24898 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: TxnStatusTablet state changed. Reason: New leader 0d244af8860e40debe852d5b48d17141. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.779268 24898 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.780903 24915 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.781045 24915 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:22.783337 24915 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.783447 24915 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 0d244af8860e40debe852d5b48d17141: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:22.784024 24898 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.784147 24898 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.784200 24899 tablet_replica.cc:439] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d244af8860e40debe852d5b48d17141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 } } peers { permanent_uuid: "0d244af8860e40debe852d5b48d17141" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40393 } } peers { permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 } } }
I20260811 04:41:22.784312 24899 tablet_replica.cc:442] T 40a7f857659f4e59924157adbd043169 P e053154c0fe64a8e998b8c5930edeeb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:22.801106 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.794109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:52447 (local address 127.22.144.130:40393)
0811 04:41:22.794249 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.794254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.797194 (+ 2940us) server_negotiation.cc:407] Connection header received
0811 04:41:22.797277 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.797280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.797336 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.797424 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:22.797707 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.798564 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.799815 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.800187 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.800300 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.800635 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:22.800728 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:22.800807 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.800894 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.800950 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":49}
I20260811 04:41:22.801105 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.793978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:40393 (local address 127.22.144.130:52447)
0811 04:41:22.797093 (+ 3115us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:22.797106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:22.797202 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:22.797438 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:22.797440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:22.797446 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:22.797597 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.797602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.798717 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.798722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.799689 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:22.799700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.800317 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.800319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:22.800437 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.800489 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:22.800817 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:22.800822 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 04:41:22.800869 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 04:41:22.800938 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3017,"mutex_wait_us":30,"thread_start_us":140,"threads_started":1}
I20260811 04:41:22.809090 24819 raft_consensus.cc:1261] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Refusing update from remote peer 0d244af8860e40debe852d5b48d17141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:22.809113 24507 raft_consensus.cc:1261] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Refusing update from remote peer 0d244af8860e40debe852d5b48d17141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:22.809507 24905 consensus_queue.cc:1035] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d6e9a0ce7e945aaa4dc170b9ff95ed0" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:22.809697 24901 consensus_queue.cc:1035] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e053154c0fe64a8e998b8c5930edeeb8" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:22.821074 24911 mvcc.cc:205] Tried to move back new op lower bound from 7317189766381780992 to 7317189765401268224. Current Snapshot: MvccSnapshot[applied={T|T < 7317189766381780992}]
I20260811 04:41:22.855403 24338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:22.855585 24338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:22.857798 24338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0af5159d89494d35b5b31510a0b9a284: Sending DeleteTablet for 3 replicas of tablet 633e1fd8a2284b4f852a6300c48b9b0e
I20260811 04:41:22.858444 24487 tablet_service.cc:1514] Processing DeleteTablet for tablet 633e1fd8a2284b4f852a6300c48b9b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:45692
I20260811 04:41:22.858469 24640 tablet_service.cc:1514] Processing DeleteTablet for tablet 633e1fd8a2284b4f852a6300c48b9b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54400
I20260811 04:41:22.858690 24919 tablet_replica.cc:330] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: stopping tablet replica
I20260811 04:41:22.858814 24919 raft_consensus.cc:2229] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:22.858937 24919 raft_consensus.cc:2258] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:22.859087 24921 tablet_replica.cc:330] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: stopping tablet replica
I20260811 04:41:22.859050 24797 tablet_service.cc:1514] Processing DeleteTablet for tablet 633e1fd8a2284b4f852a6300c48b9b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54484
I20260811 04:41:22.859195 24921 raft_consensus.cc:2229] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:22.859414 24921 raft_consensus.cc:2258] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:22.859572 24952 tablet_replica.cc:330] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: stopping tablet replica
I20260811 04:41:22.859741 24952 raft_consensus.cc:2229] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:22.859992 24952 raft_consensus.cc:2258] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:22.860545 24921 ts_tablet_manager.cc:1905] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:22.860862 24919 ts_tablet_manager.cc:1905] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:22.860992 24952 ts_tablet_manager.cc:1905] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:22.863773 24921 ts_tablet_manager.cc:1918] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:22.863814 24919 ts_tablet_manager.cc:1918] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:22.863878 24921 log.cc:1196] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/633e1fd8a2284b4f852a6300c48b9b0e
I20260811 04:41:22.863890 24919 log.cc:1196] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/633e1fd8a2284b4f852a6300c48b9b0e
I20260811 04:41:22.864135 24952 ts_tablet_manager.cc:1918] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:22.864161 24919 ts_tablet_manager.cc:1939] T 633e1fd8a2284b4f852a6300c48b9b0e P e053154c0fe64a8e998b8c5930edeeb8: Deleting consensus metadata
I20260811 04:41:22.864171 24921 ts_tablet_manager.cc:1939] T 633e1fd8a2284b4f852a6300c48b9b0e P 0d244af8860e40debe852d5b48d17141: Deleting consensus metadata
I20260811 04:41:22.864283 24952 log.cc:1196] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/633e1fd8a2284b4f852a6300c48b9b0e
I20260811 04:41:22.864622 24952 ts_tablet_manager.cc:1939] T 633e1fd8a2284b4f852a6300c48b9b0e P 2d6e9a0ce7e945aaa4dc170b9ff95ed0: Deleting consensus metadata
I20260811 04:41:22.865188 24322 catalog_manager.cc:4841] TS e053154c0fe64a8e998b8c5930edeeb8 (127.22.144.129:38117): tablet 633e1fd8a2284b4f852a6300c48b9b0e (table test-table [id=5e7f97977e9340cbb61b7b29dd95d476]) successfully deleted
I20260811 04:41:22.865401 24323 catalog_manager.cc:4841] TS 0d244af8860e40debe852d5b48d17141 (127.22.144.130:40393): tablet 633e1fd8a2284b4f852a6300c48b9b0e (table test-table [id=5e7f97977e9340cbb61b7b29dd95d476]) successfully deleted
I20260811 04:41:22.865402 24321 catalog_manager.cc:4841] TS 2d6e9a0ce7e945aaa4dc170b9ff95ed0 (127.22.144.131:42743): tablet 633e1fd8a2284b4f852a6300c48b9b0e (table test-table [id=5e7f97977e9340cbb61b7b29dd95d476]) successfully deleted
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:22.880647 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.870179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.22.144.129:38117)
0811 04:41:22.870421 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.870425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.870715 (+ 290us) server_negotiation.cc:407] Connection header received
0811 04:41:22.871005 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.871009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.871067 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.871170 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:22.871913 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.872463 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.873303 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.873628 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.873740 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.876600 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:22.876619 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:22.876634 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:22.876663 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:22.878635 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.879140 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.879147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.879148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.879234 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.879531 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.879535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.879537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.879819 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:22.880044 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.880253 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.880442 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":57,"threads_started":1}
W20260811 04:41:22.881289 24550 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45700
I20260811 04:41:22.891987 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:22.883262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.22.144.190:42369)
0811 04:41:22.883409 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:22.883414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:22.883478 (+ 64us) server_negotiation.cc:407] Connection header received
0811 04:41:22.883633 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:22.883637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:22.883692 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:22.883784 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:22.884446 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.885003 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.885906 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:22.886159 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:22.886278 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:22.887725 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:22.887729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:22.887732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:22.887769 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:22.889922 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.890515 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.890519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.890521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.890620 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:22.890944 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:22.890949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:22.890951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:22.891190 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:22.891344 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:22.891641 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 04:41:22.891805 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 04:41:22.892514 24338 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48504
I20260811 04:41:22.893887 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 24417
I20260811 04:41:22.904845 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 24566
I20260811 04:41:22.914733 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 24725
I20260811 04:41:22.924623 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 24304
2026-08-11T04:41:22Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3124 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24961](info): set up 2 sockets
Aug 11 04:41:22 dist-test-slave-ss4n krb5kdc[24961](info): commencing operation
krb5kdc: starting...
W20260811 04:41:24.088882 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:24 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:24.238560 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42973
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:24.364148 24975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:24.364471 24975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:24.364522 24975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:24.368830 24975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:24.368914 24975 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 04:41:24.368932 24975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:24.368952 24975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:24.368969 24975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:24.368985 24975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:24.374773 24975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42973
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:24.375981 24975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:24.377053 24975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:24.386235 24983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:24.386257 24981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:24.386212 24980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:24.386636 24975 server_base.cc:1034] running on GCE node
I20260811 04:41:24.387368 24975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:24.388543 24975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:24.389744 24975 hybrid_clock.cc:648] HybridClock initialized: now 1786423284389727 us; error 30 us; skew 500 ppm
Aug 11 04:41:24 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:24.392880 24975 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:24.394052 24975 webserver.cc:460] Webserver started at http://127.22.144.190:38651/ using document root <none> and password file <none>
I20260811 04:41:24.394726 24975 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:24.394827 24975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:24.395068 24975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:24.397315 24975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "315f1b6166c84a20bc6bea50a3c1acfb"
format_stamp: "Formatted at 2026-08-11 04:41:24 on dist-test-slave-ss4n"
server_key: "70f153ef4a83ed11717bf29f8ca287b9"
server_key_iv: "be895f54dcce8f7667e2bf5d479e77e8"
server_key_version: "encryptionkey@0"
I20260811 04:41:24.397995 24975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "315f1b6166c84a20bc6bea50a3c1acfb"
format_stamp: "Formatted at 2026-08-11 04:41:24 on dist-test-slave-ss4n"
server_key: "70f153ef4a83ed11717bf29f8ca287b9"
server_key_iv: "be895f54dcce8f7667e2bf5d479e77e8"
server_key_version: "encryptionkey@0"
I20260811 04:41:24.402159 24975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:24.404837 24990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:24.406500 24975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:24.406689 24975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "315f1b6166c84a20bc6bea50a3c1acfb"
format_stamp: "Formatted at 2026-08-11 04:41:24 on dist-test-slave-ss4n"
server_key: "70f153ef4a83ed11717bf29f8ca287b9"
server_key_iv: "be895f54dcce8f7667e2bf5d479e77e8"
server_key_version: "encryptionkey@0"
I20260811 04:41:24.406822 24975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:24.468605 24975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:24.472681 24975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:24.472959 24975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:24.482928 24975 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:42973
I20260811 04:41:24.482962 25057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:42973 every 8 connection(s)
I20260811 04:41:24.483987 24975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:24.487309 25058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:24.488867 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 24975
I20260811 04:41:24.489037 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:24.489359 23106 external_mini_cluster.cc:1428] Setting key 5adb79c560a9c73b5b51d8b5a688ad93
I20260811 04:41:24.494774 25058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Bootstrap starting.
I20260811 04:41:24.497803 25058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:24.498826 25058 log.cc:824] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:41:24 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:24.501593 25058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: No bootstrap required, opened a new log
I20260811 04:41:24.506465 25058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42973 } }
I20260811 04:41:24.506767 25058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:24.506801 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:24.491084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.22.144.190:42973)
0811 04:41:24.491571 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:24.491595 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:24.491644 (+ 49us) server_negotiation.cc:407] Connection header received
0811 04:41:24.492603 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:24.492629 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:24.493108 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:24.493558 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:24.494305 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.495934 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.496925 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.497755 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.497949 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:24.501104 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:24.501111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:24.501128 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:24.501163 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:24.503442 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:24.503932 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:24.503938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:24.503945 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:24.504107 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:24.504418 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:24.504421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:24.504423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:24.504961 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:24.505210 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:24.505423 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 04:41:24.505774 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":121,"threads_started":1}
I20260811 04:41:24.506824 25058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315f1b6166c84a20bc6bea50a3c1acfb, State: Initialized, Role: FOLLOWER
I20260811 04:41:24.507302 25058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42973 } }
I20260811 04:41:24.507476 25058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:24.507586 25058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:24.507687 25058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:24.508749 25058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42973 } }
I20260811 04:41:24.509146 25058 leader_election.cc:302] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 315f1b6166c84a20bc6bea50a3c1acfb; no voters:
I20260811 04:41:24.509543 25058 leader_election.cc:288] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:24.509752 25063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:24.510102 25063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [term 1 LEADER]: Becoming Leader. State: Replica: 315f1b6166c84a20bc6bea50a3c1acfb, State: Running, Role: LEADER
I20260811 04:41:24.510430 25063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42973 } }
I20260811 04:41:24.510982 25058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:24.512497 25065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 315f1b6166c84a20bc6bea50a3c1acfb. Latest consensus state: current_term: 1 leader_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42973 } } }
I20260811 04:41:24.512555 25064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315f1b6166c84a20bc6bea50a3c1acfb" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42973 } } }
I20260811 04:41:24.512733 25065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:24.512773 25064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:24.513240 25072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:24.516405 25072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:24.522920 25072 catalog_manager.cc:1347] Generated new cluster ID: 9105428516284bf883bd1e3ef2317124
I20260811 04:41:24.523008 25072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:24.547024 25072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:24.548451 25072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:24.557801 25072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Generated new TSK 0
I20260811 04:41:24.558743 25072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:24.605036 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42973
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:24.728020 25086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:24.728343 25086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:24.728396 25086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:24.732800 25086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:24.732879 25086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:24.732928 25086 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 04:41:24.732944 25086 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:24.733035 25086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:24.738727 25086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:42973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:24.739959 25086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:24.741108 25086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:24.752344 25093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:24.752344 25092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:24.752349 25096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:24.752892 25086 server_base.cc:1034] running on GCE node
I20260811 04:41:24.753394 25086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:24.754160 25086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:24.755373 25086 hybrid_clock.cc:648] HybridClock initialized: now 1786423284755348 us; error 39 us; skew 500 ppm
Aug 11 04:41:24 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:24.763063 25086 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:24.764225 25086 webserver.cc:460] Webserver started at http://127.22.144.129:42035/ using document root <none> and password file <none>
I20260811 04:41:24.764928 25086 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:24.765053 25086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:24.765331 25086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:24.767941 25086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "31c9ff69e9da4def8f3b95c3a5c042ac"
format_stamp: "Formatted at 2026-08-11 04:41:24 on dist-test-slave-ss4n"
server_key: "4ed75d8b961ebdc4db413034b11722c6"
server_key_iv: "53c8fe2664780efbfcb8c6229c5da6a7"
server_key_version: "encryptionkey@0"
I20260811 04:41:24.768632 25086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "31c9ff69e9da4def8f3b95c3a5c042ac"
format_stamp: "Formatted at 2026-08-11 04:41:24 on dist-test-slave-ss4n"
server_key: "4ed75d8b961ebdc4db413034b11722c6"
server_key_iv: "53c8fe2664780efbfcb8c6229c5da6a7"
server_key_version: "encryptionkey@0"
I20260811 04:41:24.773131 25086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:24.776089 25104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:24.777657 25086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 04:41:24.777827 25086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "31c9ff69e9da4def8f3b95c3a5c042ac"
format_stamp: "Formatted at 2026-08-11 04:41:24 on dist-test-slave-ss4n"
server_key: "4ed75d8b961ebdc4db413034b11722c6"
server_key_iv: "53c8fe2664780efbfcb8c6229c5da6a7"
server_key_version: "encryptionkey@0"
I20260811 04:41:24.777972 25086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:24.800585 25086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:24.804396 25086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:24.804630 25086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:24.806340 25086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:24.806416 25086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:24.806489 25086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:24.806542 25086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:24 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:24.818020 25086 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:42655
I20260811 04:41:24.818190 25230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:42655 every 8 connection(s)
I20260811 04:41:24.818975 25086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:24.821434 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:24.807184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:34797 (local address 127.22.144.190:42973)
0811 04:41:24.807363 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:24.807370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:24.808282 (+ 912us) server_negotiation.cc:407] Connection header received
0811 04:41:24.809661 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:24.809668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:24.809737 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:24.809802 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:24.812029 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.812633 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.814150 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.814407 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.814538 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:24.817666 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:24.817670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:24.817673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:24.817709 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:24.819327 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:24.819965 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:24.819969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:24.819971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:24.820076 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:24.820381 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:24.820384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:24.820385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:24.820571 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:24.820646 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:24.821174 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 04:41:24.821298 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:41:24.822204 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:24.807593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.22.144.129:34797)
0811 04:41:24.808087 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:24.808149 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:24.809344 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:24.810291 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:24.810304 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:24.810946 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:24.811809 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:24.811833 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.812781 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.812785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:24.814018 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:24.814028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.814588 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.814592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:24.814778 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:24.815431 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:24.815448 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:24.817422 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:24.819458 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:24.819464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:24.819481 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:24.819837 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:24.820158 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:24.820160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:24.820162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:24.820289 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:24.820668 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:24.820673 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:24.820899 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 04:41:24.821050 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 04:41:24.821303 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260811 04:41:24.823655 25231 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42973
I20260811 04:41:24.823815 25231 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:24.824322 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 25086
I20260811 04:41:24.824379 25231 heartbeater.cc:511] Master 127.22.144.190:42973 requested a full tablet report, sending...
I20260811 04:41:24.824448 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:24.824801 23106 external_mini_cluster.cc:1428] Setting key 64fd77a1bc3497eef16b1a1e9b3d08ec
I20260811 04:41:24.826145 25004 ts_manager.cc:194] Registered new tserver with Master: 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655)
I20260811 04:41:24.827756 25004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:34797
I20260811 04:41:24.834779 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:24.826272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:51333 (local address 127.22.144.190:42973)
0811 04:41:24.826541 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:24.826546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:24.826748 (+ 202us) server_negotiation.cc:407] Connection header received
0811 04:41:24.826863 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:24.826867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:24.826937 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:24.827036 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:24.827552 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.828346 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.829470 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.829748 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.829875 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:24.831396 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:24.831400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:24.831402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:24.831425 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:24.833057 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:24.833554 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:24.833557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:24.833558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:24.833636 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:24.833932 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:24.833935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:24.833937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:24.834124 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:24.834236 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:24.834498 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 04:41:24.834609 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":105}
I20260811 04:41:24.834797 25237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:24.826308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.22.144.129:51333)
0811 04:41:24.826631 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:24.826646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:24.826761 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:24.827073 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:24.827076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:24.827301 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:24.827424 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:24.827431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.828567 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.828574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:24.829325 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:24.829336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:24.829882 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:24.829886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:24.829998 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:24.830660 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:24.830682 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:24.831248 (+ 566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:24.833178 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:24.833183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:24.833185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:24.833398 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:24.833733 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:24.833737 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:24.833739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:24.833831 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:24.834226 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:24.834229 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:24.834331 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 04:41:24.834443 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 04:41:24.834604 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":23,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:24.860818 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42973
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:24.979894 25241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:24.980218 25241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:24.980305 25241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:24.984778 25241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:24.984871 25241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:24.984941 25241 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 04:41:24.984983 25241 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:24.985052 25241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:24.990620 25241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:42973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:24.991783 25241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:24.992960 25241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:25.003631 25247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:25.003645 25246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:25.003898 25249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:25.003669 25241 server_base.cc:1034] running on GCE node
I20260811 04:41:25.004381 25241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:25.005117 25241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:25.006299 25241 hybrid_clock.cc:648] HybridClock initialized: now 1786423285006246 us; error 63 us; skew 500 ppm
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:25.009851 25241 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:25.011041 25241 webserver.cc:460] Webserver started at http://127.22.144.130:38443/ using document root <none> and password file <none>
I20260811 04:41:25.011859 25241 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:25.011966 25241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:25.012192 25241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:25.014744 25241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "f825dcd8407446f08fce565fcbae6890"
format_stamp: "Formatted at 2026-08-11 04:41:25 on dist-test-slave-ss4n"
server_key: "b0b3aa984a103aa816c96426ca0dff67"
server_key_iv: "a90a69417e4e526de84c48de3ac9b386"
server_key_version: "encryptionkey@0"
I20260811 04:41:25.015316 25241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "f825dcd8407446f08fce565fcbae6890"
format_stamp: "Formatted at 2026-08-11 04:41:25 on dist-test-slave-ss4n"
server_key: "b0b3aa984a103aa816c96426ca0dff67"
server_key_iv: "a90a69417e4e526de84c48de3ac9b386"
server_key_version: "encryptionkey@0"
I20260811 04:41:25.019515 25241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:25.022328 25256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:25.023944 25241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:25.024108 25241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "f825dcd8407446f08fce565fcbae6890"
format_stamp: "Formatted at 2026-08-11 04:41:25 on dist-test-slave-ss4n"
server_key: "b0b3aa984a103aa816c96426ca0dff67"
server_key_iv: "a90a69417e4e526de84c48de3ac9b386"
server_key_version: "encryptionkey@0"
I20260811 04:41:25.024276 25241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:25.044304 25241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:25.047895 25241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:25.048141 25241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:25.049966 25241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:25.050034 25241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:25.050078 25241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:25.050132 25241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:25.062638 25241 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:43253
I20260811 04:41:25.062738 25382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:43253 every 8 connection(s)
I20260811 04:41:25.063983 25241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:25.066921 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.051075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:34409 (local address 127.22.144.190:42973)
0811 04:41:25.051230 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.051234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.052437 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 04:41:25.053741 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.053745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.053810 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.053925 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:25.055715 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.056719 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.058773 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.059159 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.059287 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.062519 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:25.062526 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:25.062531 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:25.062571 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:25.064594 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.065247 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.065251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.065252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.065342 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.065768 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.065770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.065772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.065973 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:25.066062 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.066674 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.066794 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:41:25.067782 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.051622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.22.144.130:34409)
0811 04:41:25.052255 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.052305 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.053377 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.054126 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.054144 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.054751 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:25.055487 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.055509 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.056886 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.056890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.058595 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.058610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.059347 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.059350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.059504 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.060425 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:25.060452 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:25.062290 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:25.064751 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.064760 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.064778 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.065117 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.065435 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.065437 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.065439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.065663 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.066107 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:25.066114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:25.066368 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.066567 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.066856 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":154,"threads_started":1}
I20260811 04:41:25.069164 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 25241
I20260811 04:41:25.069283 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:25.069249 25383 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42973
I20260811 04:41:25.069386 25383 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:25.069657 23106 external_mini_cluster.cc:1428] Setting key 9a9980b2603a10823ce34e0ce027d54d
I20260811 04:41:25.069976 25383 heartbeater.cc:511] Master 127.22.144.190:42973 requested a full tablet report, sending...
I20260811 04:41:25.071138 25004 ts_manager.cc:194] Registered new tserver with Master: f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:25.071967 25004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:34409
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:41:25.082712 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.072410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.22.144.130:53411)
0811 04:41:25.073662 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.073677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.073832 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.074193 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.074196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.074426 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:25.074541 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.074549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.075748 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.075754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.076805 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.076821 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.077350 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.077354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.077471 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.078168 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:25.078191 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:25.078761 (+ 570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:25.080756 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.080762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.080764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.080995 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.081327 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.081331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.081333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.081432 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.082037 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:25.082040 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:25.082175 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.082295 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.082476 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":110,"threads_started":1}
I20260811 04:41:25.082705 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.072187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:53411 (local address 127.22.144.190:42973)
0811 04:41:25.072314 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.072320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.073748 (+ 1428us) server_negotiation.cc:407] Connection header received
0811 04:41:25.073944 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.073947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.073992 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.074056 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:25.074734 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.075592 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.076974 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.077210 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.077339 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.078967 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:25.078973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:25.078978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:25.079015 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:25.080629 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.081117 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.081121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.081123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.081212 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.081555 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.081567 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.081569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.081918 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:25.082049 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.082372 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.082506 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:25.105901 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42973
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:25.229358 25393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:25.229715 25393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:25.229768 25393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:25.233814 25393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:25.233878 25393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:25.233919 25393 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 04:41:25.233933 25393 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:25.233965 25393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:25.239264 25393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:42973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:25.240347 25393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:25.241351 25393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:25.251890 25399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:25.251902 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:25.252100 25393 server_base.cc:1034] running on GCE node
W20260811 04:41:25.251921 25402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:25.252705 25393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:25.253440 25393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:25.254665 25393 hybrid_clock.cc:648] HybridClock initialized: now 1786423285254629 us; error 55 us; skew 500 ppm
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:25.257833 25393 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:25.258816 25393 webserver.cc:460] Webserver started at http://127.22.144.131:40149/ using document root <none> and password file <none>
I20260811 04:41:25.259405 25393 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:25.259500 25393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:25.259743 25393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:25.261951 25393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "f16551beeb214459a107a365b85e89a7"
format_stamp: "Formatted at 2026-08-11 04:41:25 on dist-test-slave-ss4n"
server_key: "bf05406dac7e35dbe4a78e2540756738"
server_key_iv: "8fe24aea956aaeb9dd8678ceaab800ac"
server_key_version: "encryptionkey@0"
I20260811 04:41:25.262562 25393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "f16551beeb214459a107a365b85e89a7"
format_stamp: "Formatted at 2026-08-11 04:41:25 on dist-test-slave-ss4n"
server_key: "bf05406dac7e35dbe4a78e2540756738"
server_key_iv: "8fe24aea956aaeb9dd8678ceaab800ac"
server_key_version: "encryptionkey@0"
I20260811 04:41:25.266781 25393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:41:25.270112 25409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:25.271658 25393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:41:25.271829 25393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "f16551beeb214459a107a365b85e89a7"
format_stamp: "Formatted at 2026-08-11 04:41:25 on dist-test-slave-ss4n"
server_key: "bf05406dac7e35dbe4a78e2540756738"
server_key_iv: "8fe24aea956aaeb9dd8678ceaab800ac"
server_key_version: "encryptionkey@0"
I20260811 04:41:25.271978 25393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:25.314671 25393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:25.318645 25393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:25.318894 25393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:25.320640 25393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:25.320719 25393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:25.320797 25393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:25.320858 25393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:25.333400 25393 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:44315
I20260811 04:41:25.333536 25536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:44315 every 8 connection(s)
I20260811 04:41:25.334463 25393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:25.335302 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 25393
I20260811 04:41:25.335438 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:25.335825 23106 external_mini_cluster.cc:1428] Setting key 952f6a4786541ff1ce8da40f6a5f4d12
I20260811 04:41:25.338814 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.321777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:56971 (local address 127.22.144.190:42973)
0811 04:41:25.321924 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.321928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.324102 (+ 2174us) server_negotiation.cc:407] Connection header received
0811 04:41:25.325842 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.325846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.325907 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.326045 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:25.327486 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.328148 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.330234 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.330458 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.330571 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.334144 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:25.334148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:25.334151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:25.334192 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:25.336096 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.336930 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.336934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.336936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.337055 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.337549 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.337554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.337556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.337776 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:25.337866 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.338571 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.338689 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:41:25.339705 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.322462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.22.144.131:56971)
0811 04:41:25.323823 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.323978 (+ 155us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.325417 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.326231 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.326243 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.326658 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:25.327329 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.327347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.328290 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.328295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.330075 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.330088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.330652 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.330658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.330916 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.331700 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:25.331732 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:25.333943 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:25.336226 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.336233 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.336251 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.336786 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.337168 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.337172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.337174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.337375 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.337901 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:25.337908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:25.338258 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.338480 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.338778 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"thread_start_us":143,"threads_started":1}
I20260811 04:41:25.340866 25537 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42973
I20260811 04:41:25.341029 25537 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:25.341684 25537 heartbeater.cc:511] Master 127.22.144.190:42973 requested a full tablet report, sending...
I20260811 04:41:25.342957 25005 ts_manager.cc:194] Registered new tserver with Master: f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315)
I20260811 04:41:25.343823 25005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:56971
I20260811 04:41:25.350644 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:41:25.351804 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.343700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:47751 (local address 127.22.144.190:42973)
0811 04:41:25.343832 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.343835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.344036 (+ 201us) server_negotiation.cc:407] Connection header received
0811 04:41:25.344160 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.344166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.344217 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.344302 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:25.344831 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.345343 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.346198 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.346434 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.346559 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.347861 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:25.347865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:25.347868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:25.347902 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:25.349746 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.350332 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.350336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.350338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.350449 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.350791 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.350795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.350797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.351013 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:25.351100 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.351496 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.351602 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 04:41:25.351804 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.343614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.22.144.131:47751)
0811 04:41:25.343954 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.343970 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.344072 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.344337 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.344340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.344607 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:25.344708 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.344714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.345473 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.345476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.346072 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.346080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.346565 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.346569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.346660 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.347221 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:25.347254 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:25.347686 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:25.349901 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.349904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.349934 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.350224 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.350573 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.350577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.350578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.350680 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.351122 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:25.351126 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:25.351241 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.351362 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.351525 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":97,"threads_started":1}
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:25.382643 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.372782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.22.144.190:42973)
0811 04:41:25.372935 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.372939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.372953 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:25.372998 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.373001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.373044 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.373180 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:25.374086 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.374665 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.375676 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.375925 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.376046 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.379074 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:25.379078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:25.379080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:25.379113 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:25.380628 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.381132 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.381143 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.381144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.381228 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.381575 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.381582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.381587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.381859 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:25.382022 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.382319 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.382459 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:41:25.386160 25005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:25.389351 25005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:25.405400 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.399030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44315 (local address 127.0.0.1:40438)
0811 04:41:25.399893 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.399969 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.400110 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.400817 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.400823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.400859 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.401040 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.401047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.402386 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.402392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.403722 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.403736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.404797 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.404801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.404919 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.404972 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.405080 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.405209 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":41,"spinlock_wait_cycles":3456,"thread_start_us":147,"threads_started":1}
I20260811 04:41:25.405803 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.399851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42655 (local address 127.0.0.1:40442)
0811 04:41:25.400750 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.400765 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.400863 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.401607 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.401610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.401630 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.401722 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.401728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.403119 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.403122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.404231 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.404239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.405121 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.405127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.405221 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.405246 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.405322 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.405400 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"spinlock_wait_cycles":1280,"thread_start_us":97,"threads_started":1}
I20260811 04:41:25.405839 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.400028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40442 (local address 127.22.144.129:42655)
0811 04:41:25.400419 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.400426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.400971 (+ 545us) server_negotiation.cc:407] Connection header received
0811 04:41:25.401052 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.401058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.401236 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.401396 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.401848 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.402989 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.404345 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.404980 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.405124 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.405247 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.405346 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.405451 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":86,"threads_started":1}
I20260811 04:41:25.405908 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.399141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.22.144.131:44315)
0811 04:41:25.400256 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.400271 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.400298 (+ 27us) server_negotiation.cc:407] Connection header received
0811 04:41:25.400380 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.400386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.400637 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.400836 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.401167 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.402266 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.403874 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.404662 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.404808 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.404957 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.405660 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.405773 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":109,"threads_started":1}
I20260811 04:41:25.406386 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.399095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43253 (local address 127.0.0.1:51534)
0811 04:41:25.400567 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.400581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.400671 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.401199 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.401202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.401225 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.401315 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.401320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.402725 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.402728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.404046 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.404057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.405393 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.405396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.405641 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.405664 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.405736 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.405800 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1413,"mutex_wait_us":27,"thread_start_us":130,"threads_started":1}
I20260811 04:41:25.406580 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.400244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.22.144.130:43253)
0811 04:41:25.400701 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.400713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.400737 (+ 24us) server_negotiation.cc:407] Connection header received
0811 04:41:25.400821 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.400827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.401044 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.401202 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.401435 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.402583 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.404189 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.405062 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.405203 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.405332 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.406233 (+ 901us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.406337 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":124,"threads_started":1}
I20260811 04:41:25.408743 25465 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc1626cbd340e1bcaa1f0fa265181c (DEFAULT_TABLE table=test-table [id=79c9e353b80c40f4b52769b9082be9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:25.409205 25312 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc1626cbd340e1bcaa1f0fa265181c (DEFAULT_TABLE table=test-table [id=79c9e353b80c40f4b52769b9082be9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:25.409198 25158 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc1626cbd340e1bcaa1f0fa265181c (DEFAULT_TABLE table=test-table [id=79c9e353b80c40f4b52769b9082be9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:25.409888 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ffc1626cbd340e1bcaa1f0fa265181c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.410245 25312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ffc1626cbd340e1bcaa1f0fa265181c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.410259 25158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ffc1626cbd340e1bcaa1f0fa265181c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.417680 25563 tablet_bootstrap.cc:492] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Bootstrap starting.
I20260811 04:41:25.418603 25564 tablet_bootstrap.cc:492] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Bootstrap starting.
I20260811 04:41:25.419062 25565 tablet_bootstrap.cc:492] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Bootstrap starting.
I20260811 04:41:25.420535 25564 tablet_bootstrap.cc:654] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.420634 25563 tablet_bootstrap.cc:654] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.421010 25565 tablet_bootstrap.cc:654] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.421419 25564 log.cc:824] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:25.421804 25565 log.cc:824] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:25.422186 25563 log.cc:824] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:25.423667 25564 tablet_bootstrap.cc:492] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: No bootstrap required, opened a new log
I20260811 04:41:25.423736 25565 tablet_bootstrap.cc:492] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: No bootstrap required, opened a new log
I20260811 04:41:25.423883 25564 ts_tablet_manager.cc:1397] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 04:41:25.423916 25565 ts_tablet_manager.cc:1397] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 04:41:25.424247 25563 tablet_bootstrap.cc:492] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: No bootstrap required, opened a new log
I20260811 04:41:25.424415 25563 ts_tablet_manager.cc:1397] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 04:41:25.427646 25564 raft_consensus.cc:348] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.427898 25564 raft_consensus.cc:374] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.427954 25564 raft_consensus.cc:729] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.427917 25565 raft_consensus.cc:348] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.428174 25565 raft_consensus.cc:374] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.428222 25565 raft_consensus.cc:729] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825dcd8407446f08fce565fcbae6890, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.428107 25563 raft_consensus.cc:348] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.428366 25563 raft_consensus.cc:374] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.428426 25563 raft_consensus.cc:729] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16551beeb214459a107a365b85e89a7, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.428704 25565 consensus_queue.cc:260] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.428761 25563 consensus_queue.cc:260] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.428895 25564 consensus_queue.cc:260] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.429760 25565 ts_tablet_manager.cc:1428] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:41:25.429761 25563 ts_tablet_manager.cc:1428] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 04:41:25.430140 25564 ts_tablet_manager.cc:1428] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 04:41:25.430160 25383 heartbeater.cc:503] Master 127.22.144.190:42973 was elected leader, sending a full tablet report...
I20260811 04:41:25.430334 25537 heartbeater.cc:503] Master 127.22.144.190:42973 was elected leader, sending a full tablet report...
I20260811 04:41:25.430524 25231 heartbeater.cc:503] Master 127.22.144.190:42973 was elected leader, sending a full tablet report...
I20260811 04:41:25.536489 25571 raft_consensus.cc:482] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:25.536754 25571 raft_consensus.cc:504] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.538151 25571 leader_election.cc:288] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:25.542570 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.538505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:38807 (local address 127.22.144.131:44315)
0811 04:41:25.538640 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.538645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.538919 (+ 274us) server_negotiation.cc:407] Connection header received
0811 04:41:25.539187 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.539189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.539232 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.539322 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.539596 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.540143 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.541361 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.541894 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.542029 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.542096 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.542353 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.542429 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:41:25.542577 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.538711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:43111 (local address 127.22.144.130:43253)
0811 04:41:25.538873 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.538880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.539098 (+ 218us) server_negotiation.cc:407] Connection header received
0811 04:41:25.539228 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.539232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.539283 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.539372 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.539734 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.540255 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.541326 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.541863 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.542011 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.542074 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.542322 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.542404 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 04:41:25.542644 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.538372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44315 (local address 127.22.144.129:38807)
0811 04:41:25.538818 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.538834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.539011 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.539333 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.539336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.539352 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.539487 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.539492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.540259 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.540264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.541233 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.541244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.542023 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.542026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.542160 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.542212 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.542302 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.542404 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":15,"thread_start_us":130,"threads_started":1}
I20260811 04:41:25.542838 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.538648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43253 (local address 127.22.144.129:43111)
0811 04:41:25.538973 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.538988 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.539102 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.539383 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.539386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.539402 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.539593 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.539601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.540375 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.540381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.541192 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.541202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.541997 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.542001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.542109 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.542209 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.542304 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.542400 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":55,"thread_start_us":125,"threads_started":1}
I20260811 04:41:25.543562 25486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc1626cbd340e1bcaa1f0fa265181c" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16551beeb214459a107a365b85e89a7" is_pre_election: true
I20260811 04:41:25.543562 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc1626cbd340e1bcaa1f0fa265181c" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890" is_pre_election: true
I20260811 04:41:25.543888 25334 raft_consensus.cc:2454] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 0.
I20260811 04:41:25.543892 25486 raft_consensus.cc:2454] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 0.
I20260811 04:41:25.544500 25105 leader_election.cc:302] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 31c9ff69e9da4def8f3b95c3a5c042ac, f16551beeb214459a107a365b85e89a7, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:25.544824 25571 raft_consensus.cc:2781] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:25.544917 25571 raft_consensus.cc:482] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:25.544960 25571 raft_consensus.cc:3037] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:25.546120 25571 raft_consensus.cc:504] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.546469 25571 leader_election.cc:288] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 election: Requested vote from peers f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:25.546847 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc1626cbd340e1bcaa1f0fa265181c" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890"
I20260811 04:41:25.546873 25486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ffc1626cbd340e1bcaa1f0fa265181c" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16551beeb214459a107a365b85e89a7"
I20260811 04:41:25.546998 25486 raft_consensus.cc:3037] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:25.546988 25334 raft_consensus.cc:3037] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:25.548338 25486 raft_consensus.cc:2454] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 1.
I20260811 04:41:25.548338 25334 raft_consensus.cc:2454] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 1.
I20260811 04:41:25.548704 25105 leader_election.cc:302] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c9ff69e9da4def8f3b95c3a5c042ac, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:25.548893 25571 raft_consensus.cc:2781] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:25.549149 25571 raft_consensus.cc:686] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 LEADER]: Becoming Leader. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Running, Role: LEADER
I20260811 04:41:25.549547 25571 consensus_queue.cc:237] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } }
I20260811 04:41:25.553033 25004 catalog_manager.cc:5495] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129). New cstate: current_term: 1 leader_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } health_report { overall_health: HEALTHY } } }
W20260811 04:41:25.565402 25384 tablet.cc:2369] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:41:25.570859 25232 tablet.cc:2369] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:41:25.586548 25538 tablet.cc:2369] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 04:41:25 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423284, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:25.620656 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.611791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.22.144.190:42973)
0811 04:41:25.611936 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.611941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.612174 (+ 233us) server_negotiation.cc:407] Connection header received
0811 04:41:25.612362 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.612366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.612407 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.612505 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:25.613202 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.613813 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.614603 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.614824 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.614952 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.617397 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:25.617401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:25.617403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:25.617433 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:25.618840 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.619316 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.619322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.619324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.619405 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:25.619681 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:25.619686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:25.619688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:25.619874 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:25.619977 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.620309 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.620502 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":50}
I20260811 04:41:25.620656 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.611706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42973 (local address 127.0.0.1:35834)
0811 04:41:25.612051 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.612083 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.612239 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.612522 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.612525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.612957 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:25.613049 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.613056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.613924 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.613928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.614484 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.614492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.614958 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.614961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.615037 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.615677 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:25.615690 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:25.617255 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:25.618948 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.618961 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.618969 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.619216 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.619503 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:25.619506 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:25.619508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:25.619589 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:25.619991 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:25.620000 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:25.620114 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.620252 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.620445 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
I20260811 04:41:25.624955 25004 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:35834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 04:41:25.625875 25004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:25.631642 25158 tablet_service.cc:1467] Processing CreateTablet for tablet 79ab14305a59495bb2af5e283b6e3025 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9e1fbafcfabf432d87cbe4ad417c23d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:25.631615 25312 tablet_service.cc:1467] Processing CreateTablet for tablet 79ab14305a59495bb2af5e283b6e3025 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9e1fbafcfabf432d87cbe4ad417c23d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:25.631829 25465 tablet_service.cc:1467] Processing CreateTablet for tablet 79ab14305a59495bb2af5e283b6e3025 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9e1fbafcfabf432d87cbe4ad417c23d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:25.632486 25158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79ab14305a59495bb2af5e283b6e3025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.632486 25312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79ab14305a59495bb2af5e283b6e3025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.632521 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79ab14305a59495bb2af5e283b6e3025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.635624 25563 tablet_bootstrap.cc:492] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: Bootstrap starting.
I20260811 04:41:25.635682 25565 tablet_bootstrap.cc:492] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: Bootstrap starting.
I20260811 04:41:25.635710 25564 tablet_bootstrap.cc:492] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: Bootstrap starting.
I20260811 04:41:25.636754 25565 tablet_bootstrap.cc:654] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.636812 25563 tablet_bootstrap.cc:654] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.636813 25564 tablet_bootstrap.cc:654] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.638357 25565 tablet_bootstrap.cc:492] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: No bootstrap required, opened a new log
I20260811 04:41:25.638469 25565 ts_tablet_manager.cc:1397] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:25.638895 25564 tablet_bootstrap.cc:492] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: No bootstrap required, opened a new log
I20260811 04:41:25.639002 25564 ts_tablet_manager.cc:1397] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 04:41:25.639345 25565 raft_consensus.cc:348] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.639494 25565 raft_consensus.cc:374] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.639434 25564 raft_consensus.cc:348] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.639533 25565 raft_consensus.cc:729] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825dcd8407446f08fce565fcbae6890, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.639566 25564 raft_consensus.cc:374] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.639617 25564 raft_consensus.cc:729] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.639747 25564 consensus_queue.cc:260] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.639721 25565 consensus_queue.cc:260] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.640059 25564 ts_tablet_manager.cc:1428] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 04:41:25.640044 25569 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.640197 25569 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.640275 25571 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.640393 25571 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.640456 25565 ts_tablet_manager.cc:1428] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 04:41:25.641474 25563 tablet_bootstrap.cc:492] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: No bootstrap required, opened a new log
I20260811 04:41:25.641655 25563 ts_tablet_manager.cc:1397] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 04:41:25.642266 25563 raft_consensus.cc:348] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.642413 25563 raft_consensus.cc:374] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.642464 25563 raft_consensus.cc:729] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16551beeb214459a107a365b85e89a7, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.642618 25563 consensus_queue.cc:260] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.642935 25563 ts_tablet_manager.cc:1428] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 04:41:25.642899 25570 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.643059 25570 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.785770 25570 raft_consensus.cc:482] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:25.785905 25570 raft_consensus.cc:504] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.787081 25570 leader_election.cc:288] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:25.792130 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.787884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:59067 (local address 127.22.144.130:43253)
0811 04:41:25.788042 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.788047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.788185 (+ 138us) server_negotiation.cc:407] Connection header received
0811 04:41:25.788323 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.788326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.788374 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.788456 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.788806 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.789425 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.790807 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.791391 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.791507 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.791550 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.791908 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.791990 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:41:25.792130 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.787616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:47095 (local address 127.22.144.129:42655)
0811 04:41:25.787766 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.787773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.787789 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:25.787849 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.787853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.787897 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.787973 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:25.788363 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.789095 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.790690 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.791331 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.791475 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.791543 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.791908 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.791993 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:41:25.792173 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.787427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42655 (local address 127.22.144.131:47095)
0811 04:41:25.787561 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.787580 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.787692 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.788034 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.788038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.788063 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.788225 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.788237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.789224 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.789227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.790547 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.790558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.791533 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.791536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.791679 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.791771 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.791845 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.791998 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":126}
I20260811 04:41:25.792186 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.787708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43253 (local address 127.22.144.131:59067)
0811 04:41:25.788063 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.788080 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.788208 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.788549 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.788551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.788571 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:25.788684 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.788689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.789559 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.789562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.790694 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.790701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.791528 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.791533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.791675 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.791757 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.791845 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.791998 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":53,"thread_start_us":133,"threads_started":1}
I20260811 04:41:25.792869 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79ab14305a59495bb2af5e283b6e3025" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890" is_pre_election: true
I20260811 04:41:25.792958 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79ab14305a59495bb2af5e283b6e3025" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" is_pre_election: true
I20260811 04:41:25.793031 25334 raft_consensus.cc:2454] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 0.
I20260811 04:41:25.793272 25181 raft_consensus.cc:2454] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 0.
I20260811 04:41:25.793524 25410 leader_election.cc:302] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16551beeb214459a107a365b85e89a7, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:25.793874 25570 raft_consensus.cc:2781] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:25.793932 25570 raft_consensus.cc:482] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:25.793960 25570 raft_consensus.cc:3037] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:25.795042 25570 raft_consensus.cc:504] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.795403 25570 leader_election.cc:288] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 election: Requested vote from peers 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:25.796012 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79ab14305a59495bb2af5e283b6e3025" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac"
I20260811 04:41:25.795991 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79ab14305a59495bb2af5e283b6e3025" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890"
I20260811 04:41:25.796136 25181 raft_consensus.cc:3037] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:25.796137 25334 raft_consensus.cc:3037] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:25.797307 25181 raft_consensus.cc:2454] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 1.
I20260811 04:41:25.797308 25334 raft_consensus.cc:2454] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 1.
I20260811 04:41:25.797673 25410 leader_election.cc:302] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16551beeb214459a107a365b85e89a7, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:25.797838 25570 raft_consensus.cc:2781] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:25.798116 25570 raft_consensus.cc:686] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [term 1 LEADER]: Becoming Leader. State: Replica: f16551beeb214459a107a365b85e89a7, State: Running, Role: LEADER
I20260811 04:41:25.798473 25570 consensus_queue.cc:237] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.800989 25588 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: TxnStatusTablet state changed. Reason: New leader f16551beeb214459a107a365b85e89a7. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.801105 25588 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:25.801450 25590 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 04:41:25.801714 25590 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 04:41:25.802031 25005 catalog_manager.cc:5495] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f16551beeb214459a107a365b85e89a7 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:25.831408 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.827572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40440 (local address 127.22.144.131:44315)
0811 04:41:25.827712 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.827717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.827733 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:25.827814 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.827816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.827868 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.827983 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:25.828256 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.828779 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.829897 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.830127 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.830237 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.830583 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.831003 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.831133 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.831210 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.831265 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:41:25.831411 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.827309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44315 (local address 127.0.0.1:40440)
0811 04:41:25.827452 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.827469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.827596 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.827991 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.827994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.828019 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:25.828140 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.828146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.828898 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.828903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.829771 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.829782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.830254 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.830257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.830358 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.830411 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.831108 (+ 697us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.831126 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.831198 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.831295 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 04:41:25.836840 25334 raft_consensus.cc:1261] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Refusing update from remote peer f16551beeb214459a107a365b85e89a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:25.836838 25181 raft_consensus.cc:1261] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Refusing update from remote peer f16551beeb214459a107a365b85e89a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:25.837606 25570 consensus_queue.cc:1035] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:25.837881 25588 consensus_queue.cc:1035] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:25.841672 25571 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: TxnStatusTablet state changed. Reason: New leader f16551beeb214459a107a365b85e89a7. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.841820 25571 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.842157 25569 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: TxnStatusTablet state changed. Reason: New leader f16551beeb214459a107a365b85e89a7. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.842289 25569 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.845978 25570 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.846105 25570 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:25.846724 25571 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.846841 25571 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P 31c9ff69e9da4def8f3b95c3a5c042ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.847278 25593 mvcc.cc:205] Tried to move back new op lower bound from 7317189778780102656 to 7317189778633256960. Current Snapshot: MvccSnapshot[applied={T|T < 7317189778780102656}]
I20260811 04:41:25.850890 25596 mvcc.cc:205] Tried to move back new op lower bound from 7317189778780102656 to 7317189778633256960. Current Snapshot: MvccSnapshot[applied={T|T < 7317189778780102656}]
I20260811 04:41:25.855721 25588 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.855860 25588 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f16551beeb214459a107a365b85e89a7: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:25.857203 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.852608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.22.144.129:42655)
0811 04:41:25.852717 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.852722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.852736 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:25.852902 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.852905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.852969 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.853067 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:25.853328 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.853929 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.855152 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.855517 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.855610 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.855979 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.856480 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.856587 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.857011 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.857069 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 04:41:25.857643 25569 tablet_replica.cc:439] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } } }
I20260811 04:41:25.857770 25569 tablet_replica.cc:442] T 79ab14305a59495bb2af5e283b6e3025 P f825dcd8407446f08fce565fcbae6890: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:25.868688 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.865086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44315 (local address 127.22.144.129:60873)
0811 04:41:25.865409 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.865424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.865570 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.865832 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.865834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.865842 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:25.865972 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.865977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.866747 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.866750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.867396 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.867405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.867879 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.867882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.867980 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.868024 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.868369 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.868378 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.868458 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.868540 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":127,"threads_started":1}
I20260811 04:41:25.868678 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.865203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:60873 (local address 127.22.144.131:44315)
0811 04:41:25.865434 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.865441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.865508 (+ 67us) server_negotiation.cc:407] Connection header received
0811 04:41:25.865676 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.865681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.865730 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.865818 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:25.866075 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.866596 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.867525 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.867765 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.867873 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.868197 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.868280 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.868361 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.868471 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.868524 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260811 04:41:25.875850 25334 raft_consensus.cc:1261] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Refusing update from remote peer 31c9ff69e9da4def8f3b95c3a5c042ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:25.876214 25486 raft_consensus.cc:1261] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Refusing update from remote peer 31c9ff69e9da4def8f3b95c3a5c042ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:25.876832 25571 consensus_queue.cc:1035] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:25.877104 25574 consensus_queue.cc:1035] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:25.902850 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.899068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:46985 (local address 127.22.144.129:42655)
0811 04:41:25.899218 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:25.899223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:25.899237 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:25.899292 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:25.899295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:25.899356 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:25.899439 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:25.899694 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.900491 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.901543 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.901903 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.902023 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.902402 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.902487 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.902549 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:25.902670 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 04:41:25.902727 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:41:25.902923 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:25.898586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42655 (local address 127.22.144.131:46985)
0811 04:41:25.898992 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:25.899010 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:25.899128 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:25.899458 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:25.899460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:25.899466 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:25.899576 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.899580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.900622 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.900625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.901372 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:25.901383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:25.902045 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:25.902049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:25.902176 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:25.902247 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:25.902581 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:25.902589 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 04:41:25.902663 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 04:41:25.902760 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":138,"threads_started":1}
I20260811 04:41:25.939299 25004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:25.939525 25004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:25.942750 25004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Sending DeleteTablet for 3 replicas of tablet 9ffc1626cbd340e1bcaa1f0fa265181c
I20260811 04:41:25.944252 25312 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc1626cbd340e1bcaa1f0fa265181c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:25 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51534
I20260811 04:41:25.944435 25004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 04:41:25.944772 25158 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc1626cbd340e1bcaa1f0fa265181c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:25 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40442
W20260811 04:41:25.944953 25004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:25.945568 25465 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc1626cbd340e1bcaa1f0fa265181c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:25 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40438
I20260811 04:41:25.946214 25616 tablet_replica.cc:330] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: stopping tablet replica
I20260811 04:41:25.946471 25617 tablet_replica.cc:330] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: stopping tablet replica
I20260811 04:41:25.946697 25618 tablet_replica.cc:330] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: stopping tablet replica
I20260811 04:41:25.947104 25618 raft_consensus.cc:2229] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:25.947502 25617 raft_consensus.cc:2229] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:25.947888 25616 raft_consensus.cc:2229] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:25.948361 25617 raft_consensus.cc:2258] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:25.948644 25616 raft_consensus.cc:2258] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:25.948863 25618 raft_consensus.cc:2258] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:25.951128 25617 ts_tablet_manager.cc:1905] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:25.951601 25312 tablet_service.cc:1467] Processing CreateTablet for tablet 48395e19a46e4b5c824626f8c5f55ac4 (DEFAULT_TABLE table=test-table [id=0dbb17fdefee48b395ccd29fbca43e07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:25.952009 25312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48395e19a46e4b5c824626f8c5f55ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.952271 25616 ts_tablet_manager.cc:1905] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:25.953770 25465 tablet_service.cc:1467] Processing CreateTablet for tablet 48395e19a46e4b5c824626f8c5f55ac4 (DEFAULT_TABLE table=test-table [id=0dbb17fdefee48b395ccd29fbca43e07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:25.954097 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48395e19a46e4b5c824626f8c5f55ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.951234 25158 tablet_service.cc:1467] Processing CreateTablet for tablet 48395e19a46e4b5c824626f8c5f55ac4 (DEFAULT_TABLE table=test-table [id=0dbb17fdefee48b395ccd29fbca43e07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:25.956981 25158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48395e19a46e4b5c824626f8c5f55ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:25.958694 25616 ts_tablet_manager.cc:1918] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:25.959219 25617 ts_tablet_manager.cc:1918] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:25.959407 25616 log.cc:1196] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/9ffc1626cbd340e1bcaa1f0fa265181c
I20260811 04:41:25.959476 25564 tablet_bootstrap.cc:492] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Bootstrap starting.
I20260811 04:41:25.960597 25564 tablet_bootstrap.cc:654] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.960845 25618 ts_tablet_manager.cc:1905] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:25.961344 25563 tablet_bootstrap.cc:492] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Bootstrap starting.
I20260811 04:41:25.961540 25617 log.cc:1196] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/9ffc1626cbd340e1bcaa1f0fa265181c
I20260811 04:41:25.961918 25616 ts_tablet_manager.cc:1939] T 9ffc1626cbd340e1bcaa1f0fa265181c P f825dcd8407446f08fce565fcbae6890: Deleting consensus metadata
I20260811 04:41:25.962368 25617 ts_tablet_manager.cc:1939] T 9ffc1626cbd340e1bcaa1f0fa265181c P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting consensus metadata
I20260811 04:41:25.962512 25563 tablet_bootstrap.cc:654] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.963752 25565 tablet_bootstrap.cc:492] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Bootstrap starting.
I20260811 04:41:25.964372 25618 ts_tablet_manager.cc:1918] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:25.964483 25618 log.cc:1196] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/9ffc1626cbd340e1bcaa1f0fa265181c
I20260811 04:41:25.964787 25565 tablet_bootstrap.cc:654] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:25.964861 25618 ts_tablet_manager.cc:1939] T 9ffc1626cbd340e1bcaa1f0fa265181c P f16551beeb214459a107a365b85e89a7: Deleting consensus metadata
I20260811 04:41:25.965549 24991 catalog_manager.cc:4841] TS 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655): tablet 9ffc1626cbd340e1bcaa1f0fa265181c (table test-table [id=79c9e353b80c40f4b52769b9082be9fb]) successfully deleted
I20260811 04:41:25.965999 24994 catalog_manager.cc:4841] TS f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315): tablet 9ffc1626cbd340e1bcaa1f0fa265181c (table test-table [id=79c9e353b80c40f4b52769b9082be9fb]) successfully deleted
I20260811 04:41:25.966092 24991 catalog_manager.cc:4841] TS f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253): tablet 9ffc1626cbd340e1bcaa1f0fa265181c (table test-table [id=79c9e353b80c40f4b52769b9082be9fb]) successfully deleted
I20260811 04:41:25.967132 25564 tablet_bootstrap.cc:492] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: No bootstrap required, opened a new log
I20260811 04:41:25.967226 25564 ts_tablet_manager.cc:1397] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260811 04:41:25.967736 25563 tablet_bootstrap.cc:492] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: No bootstrap required, opened a new log
I20260811 04:41:25.967679 25564 raft_consensus.cc:348] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.967794 25564 raft_consensus.cc:374] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.967821 25563 ts_tablet_manager.cc:1397] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 04:41:25.967823 25564 raft_consensus.cc:729] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.967943 25564 consensus_queue.cc:260] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.968202 25564 ts_tablet_manager.cc:1428] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 04:41:25.968298 25563 raft_consensus.cc:348] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.968421 25563 raft_consensus.cc:374] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.968458 25563 raft_consensus.cc:729] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16551beeb214459a107a365b85e89a7, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.968580 25563 consensus_queue.cc:260] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.968879 25563 ts_tablet_manager.cc:1428] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:25.969909 25565 tablet_bootstrap.cc:492] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: No bootstrap required, opened a new log
I20260811 04:41:25.970005 25565 ts_tablet_manager.cc:1397] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260811 04:41:25.970497 25565 raft_consensus.cc:348] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.970628 25565 raft_consensus.cc:374] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:25.970675 25565 raft_consensus.cc:729] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825dcd8407446f08fce565fcbae6890, State: Initialized, Role: FOLLOWER
I20260811 04:41:25.970805 25565 consensus_queue.cc:260] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:25.971071 25565 ts_tablet_manager.cc:1428] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 04:41:26.068718 25602 raft_consensus.cc:482] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:26.068898 25602 raft_consensus.cc:504] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:26.069469 25602 leader_election.cc:288] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:26.069974 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48395e19a46e4b5c824626f8c5f55ac4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890" is_pre_election: true
I20260811 04:41:26.070127 25334 raft_consensus.cc:2454] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 0.
I20260811 04:41:26.070055 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48395e19a46e4b5c824626f8c5f55ac4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" is_pre_election: true
I20260811 04:41:26.070243 25181 raft_consensus.cc:2454] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 0.
I20260811 04:41:26.070461 25410 leader_election.cc:302] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16551beeb214459a107a365b85e89a7, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:26.070771 25602 raft_consensus.cc:2781] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:26.070828 25602 raft_consensus.cc:482] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:26.070863 25602 raft_consensus.cc:3037] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:26.072171 25602 raft_consensus.cc:504] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:26.072556 25602 leader_election.cc:288] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 election: Requested vote from peers 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:26.073050 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48395e19a46e4b5c824626f8c5f55ac4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890"
I20260811 04:41:26.073139 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48395e19a46e4b5c824626f8c5f55ac4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac"
I20260811 04:41:26.073225 25334 raft_consensus.cc:3037] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:26.073300 25181 raft_consensus.cc:3037] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:26.074494 25334 raft_consensus.cc:2454] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 1.
I20260811 04:41:26.074839 25410 leader_election.cc:302] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16551beeb214459a107a365b85e89a7, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:26.075003 25602 raft_consensus.cc:2781] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:26.075341 25602 raft_consensus.cc:686] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 1 LEADER]: Becoming Leader. State: Replica: f16551beeb214459a107a365b85e89a7, State: Running, Role: LEADER
I20260811 04:41:26.075486 25602 consensus_queue.cc:237] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:26.078013 25181 raft_consensus.cc:2454] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 1.
I20260811 04:41:26.079648 25004 catalog_manager.cc:5495] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f16551beeb214459a107a365b85e89a7 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:26.121111 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:26.117001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40446 (local address 127.22.144.131:44315)
0811 04:41:26.117143 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:26.117146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:26.117159 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:26.117296 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:26.117298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:26.117346 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:26.117414 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:26.117780 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:26.118344 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:26.119864 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:26.120096 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:26.120185 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:26.120565 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:26.120644 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:26.120712 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:26.120849 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 04:41:26.120917 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:41:26.132341 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:26.128469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:40415 (local address 127.22.144.131:44315)
0811 04:41:26.128608 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:26.128613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:26.128629 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:26.128802 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:26.128804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:26.128850 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:26.128924 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:26.129375 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:26.130011 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:26.131281 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:26.131509 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:26.131604 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:26.131879 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:26.131969 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:26.132026 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:26.132150 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 04:41:26.132197 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:41:26.132387 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:26.128360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44315 (local address 127.22.144.131:40415)
0811 04:41:26.128560 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:26.128578 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:26.128679 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:26.128978 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:26.128984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:26.128996 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:26.129186 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:26.129216 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:26.130135 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:26.130139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:26.131133 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:26.131145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:26.131631 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:26.131635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:26.131727 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:26.131763 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:26.132055 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:26.132061 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:41:26.132130 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 04:41:26.132222 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 04:41:26.139163 25334 raft_consensus.cc:1261] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Refusing update from remote peer f16551beeb214459a107a365b85e89a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:26.139243 25181 raft_consensus.cc:1261] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Refusing update from remote peer f16551beeb214459a107a365b85e89a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:26.139632 25602 consensus_queue.cc:1035] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:26.139812 25570 consensus_queue.cc:1035] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:26.144393 25595 mvcc.cc:205] Tried to move back new op lower bound from 7317189780022185984 to 7317189779766546432. Current Snapshot: MvccSnapshot[applied={T|T < 7317189780022185984}]
W20260811 04:41:26.155334 25593 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 04:41:27.727994 25004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:27.728206 25004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:27.731473 25004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Sending DeleteTablet for 3 replicas of tablet 48395e19a46e4b5c824626f8c5f55ac4
I20260811 04:41:27.732195 25465 tablet_service.cc:1514] Processing DeleteTablet for tablet 48395e19a46e4b5c824626f8c5f55ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:27 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40438
I20260811 04:41:27.732204 25158 tablet_service.cc:1514] Processing DeleteTablet for tablet 48395e19a46e4b5c824626f8c5f55ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:27 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40442
I20260811 04:41:27.732230 25312 tablet_service.cc:1514] Processing DeleteTablet for tablet 48395e19a46e4b5c824626f8c5f55ac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:27 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51534
I20260811 04:41:27.732666 25651 tablet_replica.cc:330] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: stopping tablet replica
I20260811 04:41:27.732782 25652 tablet_replica.cc:330] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: stopping tablet replica
I20260811 04:41:27.732903 25651 raft_consensus.cc:2229] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:27.733008 25652 raft_consensus.cc:2229] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:27.733126 25651 raft_consensus.cc:2258] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:27.733234 25652 raft_consensus.cc:2258] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:27.733320 25650 tablet_replica.cc:330] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: stopping tablet replica
I20260811 04:41:27.733527 25650 raft_consensus.cc:2229] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:27.733953 25650 raft_consensus.cc:2258] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:27.734167 25651 ts_tablet_manager.cc:1905] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:27.734741 25652 ts_tablet_manager.cc:1905] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:27.734951 25650 ts_tablet_manager.cc:1905] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:27.737957 25651 ts_tablet_manager.cc:1918] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 04:41:27.737962 25650 ts_tablet_manager.cc:1918] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 04:41:27.738058 25650 log.cc:1196] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/48395e19a46e4b5c824626f8c5f55ac4
I20260811 04:41:27.738157 25651 log.cc:1196] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/48395e19a46e4b5c824626f8c5f55ac4
I20260811 04:41:27.738528 25651 ts_tablet_manager.cc:1939] T 48395e19a46e4b5c824626f8c5f55ac4 P f825dcd8407446f08fce565fcbae6890: Deleting consensus metadata
I20260811 04:41:27.738536 25650 ts_tablet_manager.cc:1939] T 48395e19a46e4b5c824626f8c5f55ac4 P f16551beeb214459a107a365b85e89a7: Deleting consensus metadata
I20260811 04:41:27.739491 24991 catalog_manager.cc:4841] TS f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253): tablet 48395e19a46e4b5c824626f8c5f55ac4 (table test-table [id=0dbb17fdefee48b395ccd29fbca43e07]) successfully deleted
I20260811 04:41:27.739491 24994 catalog_manager.cc:4841] TS f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315): tablet 48395e19a46e4b5c824626f8c5f55ac4 (table test-table [id=0dbb17fdefee48b395ccd29fbca43e07]) successfully deleted
I20260811 04:41:27.740880 25652 ts_tablet_manager.cc:1918] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 04:41:27.741067 25652 log.cc:1196] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/48395e19a46e4b5c824626f8c5f55ac4
I20260811 04:41:27.741387 25652 ts_tablet_manager.cc:1939] T 48395e19a46e4b5c824626f8c5f55ac4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting consensus metadata
I20260811 04:41:27.742252 24991 catalog_manager.cc:4841] TS 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655): tablet 48395e19a46e4b5c824626f8c5f55ac4 (table test-table [id=0dbb17fdefee48b395ccd29fbca43e07]) successfully deleted
Aug 11 04:41:27 dist-test-slave-ss4n krb5kdc[24961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 04:41:27 dist-test-slave-ss4n krb5kdc[24961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:27.766415 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:27.755947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.22.144.190:42973)
0811 04:41:27.756279 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:27.756290 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:27.756313 (+ 23us) server_negotiation.cc:407] Connection header received
0811 04:41:27.756496 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:27.756500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:27.756568 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:27.756672 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:27.757470 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:27.758090 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:27.758962 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:27.759230 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:27.759345 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:27.762525 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:27.762529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:27.762531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:27.762564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:27.764296 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:27.764863 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:27.764868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:27.764870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:27.764974 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:27.765305 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:27.765309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:27.765311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:27.765618 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:27.765777 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:27.766054 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 04:41:27.766210 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":119,"threads_started":1}
I20260811 04:41:27.769750 25004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:27.770323 25004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:27.775563 25158 tablet_service.cc:1467] Processing CreateTablet for tablet 99c8b0fd0d174dad80ecfd72775402c3 (DEFAULT_TABLE table=test-table [id=38f664ace4d04af086b084ac3ba1f320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:27.775596 25312 tablet_service.cc:1467] Processing CreateTablet for tablet 99c8b0fd0d174dad80ecfd72775402c3 (DEFAULT_TABLE table=test-table [id=38f664ace4d04af086b084ac3ba1f320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:27.775681 25465 tablet_service.cc:1467] Processing CreateTablet for tablet 99c8b0fd0d174dad80ecfd72775402c3 (DEFAULT_TABLE table=test-table [id=38f664ace4d04af086b084ac3ba1f320]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:27.775902 25158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99c8b0fd0d174dad80ecfd72775402c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:27.775935 25312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99c8b0fd0d174dad80ecfd72775402c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:27.775943 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99c8b0fd0d174dad80ecfd72775402c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:27.778764 25664 tablet_bootstrap.cc:492] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Bootstrap starting.
I20260811 04:41:27.778896 25661 tablet_bootstrap.cc:492] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Bootstrap starting.
I20260811 04:41:27.778936 25663 tablet_bootstrap.cc:492] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Bootstrap starting.
I20260811 04:41:27.779716 25663 tablet_bootstrap.cc:654] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:27.779734 25664 tablet_bootstrap.cc:654] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:27.780081 25661 tablet_bootstrap.cc:654] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:27.780982 25663 tablet_bootstrap.cc:492] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: No bootstrap required, opened a new log
I20260811 04:41:27.781102 25663 ts_tablet_manager.cc:1397] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:27.781533 25664 tablet_bootstrap.cc:492] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: No bootstrap required, opened a new log
I20260811 04:41:27.781571 25661 tablet_bootstrap.cc:492] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: No bootstrap required, opened a new log
I20260811 04:41:27.781643 25664 ts_tablet_manager.cc:1397] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:27.781689 25663 raft_consensus.cc:348] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:27.781782 25661 ts_tablet_manager.cc:1397] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:27.781816 25663 raft_consensus.cc:374] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:27.781852 25663 raft_consensus.cc:729] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Initialized, Role: FOLLOWER
I20260811 04:41:27.781994 25663 consensus_queue.cc:260] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:27.782084 25664 raft_consensus.cc:348] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:27.782220 25664 raft_consensus.cc:374] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:27.782265 25664 raft_consensus.cc:729] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16551beeb214459a107a365b85e89a7, State: Initialized, Role: FOLLOWER
I20260811 04:41:27.782347 25661 raft_consensus.cc:348] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:27.782405 25663 ts_tablet_manager.cc:1428] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 04:41:27.782471 25661 raft_consensus.cc:374] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:27.782406 25664 consensus_queue.cc:260] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:27.782732 25664 ts_tablet_manager.cc:1428] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 04:41:27.782513 25661 raft_consensus.cc:729] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825dcd8407446f08fce565fcbae6890, State: Initialized, Role: FOLLOWER
I20260811 04:41:27.783211 25661 consensus_queue.cc:260] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:27.783630 25661 ts_tablet_manager.cc:1428] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:28.138761 25668 raft_consensus.cc:482] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:28.138953 25668 raft_consensus.cc:504] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.139678 25668 leader_election.cc:288] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253), f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315)
I20260811 04:41:28.140084 25486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8b0fd0d174dad80ecfd72775402c3" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16551beeb214459a107a365b85e89a7" is_pre_election: true
I20260811 04:41:28.140120 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8b0fd0d174dad80ecfd72775402c3" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890" is_pre_election: true
I20260811 04:41:28.140291 25334 raft_consensus.cc:2454] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 0.
I20260811 04:41:28.140286 25486 raft_consensus.cc:2454] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 0.
I20260811 04:41:28.140648 25105 leader_election.cc:302] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c9ff69e9da4def8f3b95c3a5c042ac, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:28.140859 25668 raft_consensus.cc:2781] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:28.140944 25668 raft_consensus.cc:482] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:28.141000 25668 raft_consensus.cc:3037] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:28.142303 25668 raft_consensus.cc:504] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.142733 25668 leader_election.cc:288] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 election: Requested vote from peers f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253), f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315)
I20260811 04:41:28.143121 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8b0fd0d174dad80ecfd72775402c3" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890"
I20260811 04:41:28.143275 25334 raft_consensus.cc:3037] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:28.143285 25486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8b0fd0d174dad80ecfd72775402c3" candidate_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f16551beeb214459a107a365b85e89a7"
I20260811 04:41:28.143466 25486 raft_consensus.cc:3037] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:28.144273 25334 raft_consensus.cc:2454] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 1.
I20260811 04:41:28.144572 25105 leader_election.cc:302] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c9ff69e9da4def8f3b95c3a5c042ac, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:28.144629 25486 raft_consensus.cc:2454] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31c9ff69e9da4def8f3b95c3a5c042ac in term 1.
I20260811 04:41:28.144770 25668 raft_consensus.cc:2781] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:28.145058 25668 raft_consensus.cc:686] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 LEADER]: Becoming Leader. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Running, Role: LEADER
I20260811 04:41:28.145232 25668 consensus_queue.cc:237] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.147146 25005 catalog_manager.cc:5495] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129). New cstate: current_term: 1 leader_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } health_report { overall_health: UNKNOWN } } }
I20260811 04:41:28.243172 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:28.239656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40470 (local address 127.22.144.129:42655)
0811 04:41:28.239954 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:28.239963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:28.239978 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:28.240030 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:28.240032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:28.240076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:28.240149 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:28.240404 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:28.240979 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:28.241909 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:28.242132 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:28.242403 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:28.242681 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:28.242761 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:28.242852 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:28.242952 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 04:41:28.243005 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":113,"threads_started":1}
I20260811 04:41:28.250388 25334 raft_consensus.cc:1261] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Refusing update from remote peer 31c9ff69e9da4def8f3b95c3a5c042ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:28.250394 25486 raft_consensus.cc:1261] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Refusing update from remote peer 31c9ff69e9da4def8f3b95c3a5c042ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:28.250834 25668 consensus_queue.cc:1035] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:28.251017 25671 consensus_queue.cc:1035] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:28.287544 25003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:28.287719 25003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:28.289978 25003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Sending DeleteTablet for 3 replicas of tablet 99c8b0fd0d174dad80ecfd72775402c3
I20260811 04:41:28.290552 25158 tablet_service.cc:1514] Processing DeleteTablet for tablet 99c8b0fd0d174dad80ecfd72775402c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:28 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40442
I20260811 04:41:28.290750 25312 tablet_service.cc:1514] Processing DeleteTablet for tablet 99c8b0fd0d174dad80ecfd72775402c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:28 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51534
I20260811 04:41:28.290774 25465 tablet_service.cc:1514] Processing DeleteTablet for tablet 99c8b0fd0d174dad80ecfd72775402c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:28 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40438
I20260811 04:41:28.291075 25691 tablet_replica.cc:330] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: stopping tablet replica
I20260811 04:41:28.291097 25692 tablet_replica.cc:330] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: stopping tablet replica
I20260811 04:41:28.291198 25691 raft_consensus.cc:2229] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:28.291198 25692 raft_consensus.cc:2229] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:28.291311 25692 raft_consensus.cc:2258] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:28.291311 25691 raft_consensus.cc:2258] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:28.291579 25690 tablet_replica.cc:330] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: stopping tablet replica
I20260811 04:41:28.291705 25690 raft_consensus.cc:2229] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:28.291652 25003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 04:41:28.291929 25690 raft_consensus.cc:2258] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:28.292162 25691 ts_tablet_manager.cc:1905] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 04:41:28.292420 25003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:28.292565 25690 ts_tablet_manager.cc:1905] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:28.292702 25692 ts_tablet_manager.cc:1905] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:28.295024 25691 ts_tablet_manager.cc:1918] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:28.295133 25691 log.cc:1196] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/99c8b0fd0d174dad80ecfd72775402c3
I20260811 04:41:28.295421 25691 ts_tablet_manager.cc:1939] T 99c8b0fd0d174dad80ecfd72775402c3 P f16551beeb214459a107a365b85e89a7: Deleting consensus metadata
I20260811 04:41:28.295907 25692 ts_tablet_manager.cc:1918] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:28.296056 25692 log.cc:1196] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/99c8b0fd0d174dad80ecfd72775402c3
I20260811 04:41:28.296142 24994 catalog_manager.cc:4841] TS f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315): tablet 99c8b0fd0d174dad80ecfd72775402c3 (table test-table [id=38f664ace4d04af086b084ac3ba1f320]) successfully deleted
I20260811 04:41:28.296285 25690 ts_tablet_manager.cc:1918] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:28.296387 25690 log.cc:1196] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/99c8b0fd0d174dad80ecfd72775402c3
I20260811 04:41:28.296604 25692 ts_tablet_manager.cc:1939] T 99c8b0fd0d174dad80ecfd72775402c3 P f825dcd8407446f08fce565fcbae6890: Deleting consensus metadata
I20260811 04:41:28.296747 25690 ts_tablet_manager.cc:1939] T 99c8b0fd0d174dad80ecfd72775402c3 P 31c9ff69e9da4def8f3b95c3a5c042ac: Deleting consensus metadata
I20260811 04:41:28.297408 24991 catalog_manager.cc:4841] TS f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253): tablet 99c8b0fd0d174dad80ecfd72775402c3 (table test-table [id=38f664ace4d04af086b084ac3ba1f320]) successfully deleted
I20260811 04:41:28.297603 24991 catalog_manager.cc:4841] TS 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655): tablet 99c8b0fd0d174dad80ecfd72775402c3 (table test-table [id=38f664ace4d04af086b084ac3ba1f320]) successfully deleted
I20260811 04:41:28.298204 25158 tablet_service.cc:1467] Processing CreateTablet for tablet c7a9e83fa742480e9d93c8c3c57faee4 (DEFAULT_TABLE table=test-table [id=73b65c22ed694ea8a17e2a17fe75d17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:28.298334 25312 tablet_service.cc:1467] Processing CreateTablet for tablet c7a9e83fa742480e9d93c8c3c57faee4 (DEFAULT_TABLE table=test-table [id=73b65c22ed694ea8a17e2a17fe75d17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:28.298468 25158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7a9e83fa742480e9d93c8c3c57faee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:28.298475 25465 tablet_service.cc:1467] Processing CreateTablet for tablet c7a9e83fa742480e9d93c8c3c57faee4 (DEFAULT_TABLE table=test-table [id=73b65c22ed694ea8a17e2a17fe75d17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:28.298650 25312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7a9e83fa742480e9d93c8c3c57faee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:28.298735 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7a9e83fa742480e9d93c8c3c57faee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:28.300961 25693 tablet_bootstrap.cc:492] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Bootstrap starting.
I20260811 04:41:28.301155 25695 tablet_bootstrap.cc:492] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Bootstrap starting.
I20260811 04:41:28.301669 25694 tablet_bootstrap.cc:492] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Bootstrap starting.
I20260811 04:41:28.302124 25695 tablet_bootstrap.cc:654] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:28.302434 25693 tablet_bootstrap.cc:654] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:28.302565 25694 tablet_bootstrap.cc:654] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:28.303455 25695 tablet_bootstrap.cc:492] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: No bootstrap required, opened a new log
I20260811 04:41:28.303575 25695 ts_tablet_manager.cc:1397] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 04:41:28.303680 25694 tablet_bootstrap.cc:492] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac: No bootstrap required, opened a new log
I20260811 04:41:28.303774 25694 ts_tablet_manager.cc:1397] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:28.304077 25693 tablet_bootstrap.cc:492] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: No bootstrap required, opened a new log
I20260811 04:41:28.304081 25695 raft_consensus.cc:348] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.304178 25693 ts_tablet_manager.cc:1397] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 04:41:28.304147 25694 raft_consensus.cc:348] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.304224 25695 raft_consensus.cc:374] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:28.304266 25695 raft_consensus.cc:729] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16551beeb214459a107a365b85e89a7, State: Initialized, Role: FOLLOWER
I20260811 04:41:28.304273 25694 raft_consensus.cc:374] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:28.304332 25694 raft_consensus.cc:729] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31c9ff69e9da4def8f3b95c3a5c042ac, State: Initialized, Role: FOLLOWER
I20260811 04:41:28.304469 25694 consensus_queue.cc:260] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.304500 25695 consensus_queue.cc:260] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.304813 25695 ts_tablet_manager.cc:1428] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 04:41:28.305166 25693 raft_consensus.cc:348] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.305300 25693 raft_consensus.cc:374] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:28.305346 25693 raft_consensus.cc:729] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f825dcd8407446f08fce565fcbae6890, State: Initialized, Role: FOLLOWER
I20260811 04:41:28.305559 25693 consensus_queue.cc:260] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.305898 25693 ts_tablet_manager.cc:1428] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 04:41:28.306391 25694 ts_tablet_manager.cc:1428] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:28.366863 25588 raft_consensus.cc:482] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:28.367026 25588 raft_consensus.cc:504] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.367516 25588 leader_election.cc:288] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:28.368037 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a9e83fa742480e9d93c8c3c57faee4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" is_pre_election: true
I20260811 04:41:28.368096 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a9e83fa742480e9d93c8c3c57faee4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890" is_pre_election: true
I20260811 04:41:28.368244 25181 raft_consensus.cc:2454] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 0.
I20260811 04:41:28.368264 25334 raft_consensus.cc:2454] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 0.
I20260811 04:41:28.368659 25410 leader_election.cc:302] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31c9ff69e9da4def8f3b95c3a5c042ac, f16551beeb214459a107a365b85e89a7; no voters:
I20260811 04:41:28.368887 25588 raft_consensus.cc:2781] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:28.368952 25588 raft_consensus.cc:482] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:28.369007 25588 raft_consensus.cc:3037] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:28.370348 25588 raft_consensus.cc:504] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.370759 25588 leader_election.cc:288] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 election: Requested vote from peers 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655), f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253)
I20260811 04:41:28.371265 25334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a9e83fa742480e9d93c8c3c57faee4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f825dcd8407446f08fce565fcbae6890"
I20260811 04:41:28.371381 25334 raft_consensus.cc:3037] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:28.372352 25334 raft_consensus.cc:2454] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 1.
I20260811 04:41:28.372592 25181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7a9e83fa742480e9d93c8c3c57faee4" candidate_uuid: "f16551beeb214459a107a365b85e89a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac"
I20260811 04:41:28.372745 25181 raft_consensus.cc:3037] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:28.372800 25410 leader_election.cc:302] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f16551beeb214459a107a365b85e89a7, f825dcd8407446f08fce565fcbae6890; no voters:
I20260811 04:41:28.373024 25588 raft_consensus.cc:2781] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:28.373129 25588 raft_consensus.cc:686] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 1 LEADER]: Becoming Leader. State: Replica: f16551beeb214459a107a365b85e89a7, State: Running, Role: LEADER
I20260811 04:41:28.373530 25588 consensus_queue.cc:237] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } }
I20260811 04:41:28.373884 25181 raft_consensus.cc:2454] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f16551beeb214459a107a365b85e89a7 in term 1.
I20260811 04:41:28.375689 25005 catalog_manager.cc:5495] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f16551beeb214459a107a365b85e89a7 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "f16551beeb214459a107a365b85e89a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f16551beeb214459a107a365b85e89a7" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44315 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:28.391801 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:28.388701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.22.144.131:44315)
0811 04:41:28.389011 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:28.389018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:28.389031 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:28.389069 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:28.389071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:28.389110 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:28.389182 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:28.389412 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:28.390078 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:28.390904 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:28.391133 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:28.391211 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:28.391418 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:28.391496 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:28.391555 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:28.391640 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 04:41:28.391682 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":156,"threads_started":1}
I20260811 04:41:28.398219 25334 raft_consensus.cc:1261] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Refusing update from remote peer f16551beeb214459a107a365b85e89a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:28.398198 25181 raft_consensus.cc:1261] T c7a9e83fa742480e9d93c8c3c57faee4 P 31c9ff69e9da4def8f3b95c3a5c042ac [term 1 FOLLOWER]: Refusing update from remote peer f16551beeb214459a107a365b85e89a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:28.398635 25679 consensus_queue.cc:1035] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f825dcd8407446f08fce565fcbae6890" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:28.398801 25588 consensus_queue.cc:1035] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31c9ff69e9da4def8f3b95c3a5c042ac" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 04:41:28.412583 25673 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 04:41:29.965682 25004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:29.965839 25004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:29.968320 25004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 315f1b6166c84a20bc6bea50a3c1acfb: Sending DeleteTablet for 3 replicas of tablet c7a9e83fa742480e9d93c8c3c57faee4
I20260811 04:41:29.968979 25465 tablet_service.cc:1514] Processing DeleteTablet for tablet c7a9e83fa742480e9d93c8c3c57faee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:29 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40438
I20260811 04:41:29.969026 25312 tablet_service.cc:1514] Processing DeleteTablet for tablet c7a9e83fa742480e9d93c8c3c57faee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:29 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51534
I20260811 04:41:29.969349 25158 tablet_service.cc:1514] Processing DeleteTablet for tablet c7a9e83fa742480e9d93c8c3c57faee4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:29 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40442
I20260811 04:41:29.969394 25729 tablet_replica.cc:330] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: stopping tablet replica
I20260811 04:41:29.969504 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 25086
I20260811 04:41:29.969558 25729 raft_consensus.cc:2229] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:29.972373 25729 raft_consensus.cc:2258] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:29.972640 25727 tablet_replica.cc:330] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: stopping tablet replica
I20260811 04:41:29.972796 25727 raft_consensus.cc:2229] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:29.972914 25727 raft_consensus.cc:2258] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:29.973781 25727 ts_tablet_manager.cc:1905] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:29.976850 25729 ts_tablet_manager.cc:1905] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:29.977075 25727 ts_tablet_manager.cc:1918] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 04:41:29.977195 25727 log.cc:1196] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/c7a9e83fa742480e9d93c8c3c57faee4
I20260811 04:41:29.977515 25727 ts_tablet_manager.cc:1939] T c7a9e83fa742480e9d93c8c3c57faee4 P f825dcd8407446f08fce565fcbae6890: Deleting consensus metadata
I20260811 04:41:29.978230 24991 catalog_manager.cc:4841] TS f825dcd8407446f08fce565fcbae6890 (127.22.144.130:43253): tablet c7a9e83fa742480e9d93c8c3c57faee4 (table test-table [id=73b65c22ed694ea8a17e2a17fe75d17a]) successfully deleted
I20260811 04:41:29.980365 25729 ts_tablet_manager.cc:1918] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 04:41:29.980463 25729 log.cc:1196] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/c7a9e83fa742480e9d93c8c3c57faee4
I20260811 04:41:29.980785 25729 ts_tablet_manager.cc:1939] T c7a9e83fa742480e9d93c8c3c57faee4 P f16551beeb214459a107a365b85e89a7: Deleting consensus metadata
I20260811 04:41:29.981793 24994 catalog_manager.cc:4841] TS f16551beeb214459a107a365b85e89a7 (127.22.144.131:44315): tablet c7a9e83fa742480e9d93c8c3c57faee4 (table test-table [id=73b65c22ed694ea8a17e2a17fe75d17a]) successfully deleted
W20260811 04:41:29.984402 24991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:42655 (error 108)
I20260811 04:41:29.984663 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 25241
I20260811 04:41:29.988432 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:29.986040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42655 (local address 127.0.0.1:40474)
0811 04:41:29.988274 (+ 2234us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:29.988344 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:42655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2159,"thread_start_us":430,"threads_started":1}
W20260811 04:41:29.988579 24991 catalog_manager.cc:4584] TS 31c9ff69e9da4def8f3b95c3a5c042ac (127.22.144.129:42655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7a9e83fa742480e9d93c8c3c57faee4: Network error: Client connection negotiation failed: client connection to 127.22.144.129:42655: connect: Connection refused (error 111)
I20260811 04:41:29.994139 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 25393
I20260811 04:41:30.004019 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 24975
2026-08-11T04:41:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7078 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:30 dist-test-slave-ss4n krb5kdc[25734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:30 dist-test-slave-ss4n krb5kdc[25734](info): set up 2 sockets
Aug 11 04:41:30 dist-test-slave-ss4n krb5kdc[25734](info): commencing operation
krb5kdc: starting...
W20260811 04:41:31.177862 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:31 dist-test-slave-ss4n krb5kdc[25734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:31.321659 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:34005
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:34005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:31.441244 25749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:31.441592 25749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:31.441676 25749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:31.445802 25749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:31.445900 25749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:31.445940 25749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:31.445978 25749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:31.446041 25749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:31.451560 25749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:34005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:34005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:31.452850 25749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:31.454244 25749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:31.463856 25758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:31.463850 25756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:31.463856 25755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:31.463902 25749 server_base.cc:1034] running on GCE node
I20260811 04:41:31.464893 25749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:31.466140 25749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:31.467350 25749 hybrid_clock.cc:648] HybridClock initialized: now 1786423291467340 us; error 34 us; skew 500 ppm
Aug 11 04:41:31 dist-test-slave-ss4n krb5kdc[25734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:31.470494 25749 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:31.471486 25749 webserver.cc:460] Webserver started at http://127.22.144.190:33041/ using document root <none> and password file <none>
I20260811 04:41:31.472091 25749 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:31.472162 25749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:31.472424 25749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:31.474720 25749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "8b5a759d060f456ebf4ef557cbd5dde5"
format_stamp: "Formatted at 2026-08-11 04:41:31 on dist-test-slave-ss4n"
server_key: "0c7e05145a47f0406180327c518040c1"
server_key_iv: "e8dff269a4bac5f8dc0832aec24cf9bc"
server_key_version: "encryptionkey@0"
I20260811 04:41:31.475332 25749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "8b5a759d060f456ebf4ef557cbd5dde5"
format_stamp: "Formatted at 2026-08-11 04:41:31 on dist-test-slave-ss4n"
server_key: "0c7e05145a47f0406180327c518040c1"
server_key_iv: "e8dff269a4bac5f8dc0832aec24cf9bc"
server_key_version: "encryptionkey@0"
I20260811 04:41:31.479326 25749 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:31.482051 25765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:31.483402 25749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:31.483654 25749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "8b5a759d060f456ebf4ef557cbd5dde5"
format_stamp: "Formatted at 2026-08-11 04:41:31 on dist-test-slave-ss4n"
server_key: "0c7e05145a47f0406180327c518040c1"
server_key_iv: "e8dff269a4bac5f8dc0832aec24cf9bc"
server_key_version: "encryptionkey@0"
I20260811 04:41:31.483775 25749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:31.506337 25749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:31.509729 25749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:31.509984 25749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:31.518688 25824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:34005 every 8 connection(s)
I20260811 04:41:31.518689 25749 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:34005
I20260811 04:41:31.519941 25749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:31.520325 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 25749
I20260811 04:41:31.520421 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:31.520815 23106 external_mini_cluster.cc:1428] Setting key 26542f3e706dda6a4baa18567baa6aeb
I20260811 04:41:31.523706 25825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:31.530251 25825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5: Bootstrap starting.
Aug 11 04:41:31 dist-test-slave-ss4n krb5kdc[25734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:31.532886 25825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:31.533797 25825 log.cc:824] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:31.536337 25825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5: No bootstrap required, opened a new log
I20260811 04:41:31.536933 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:31.522454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.22.144.190:34005)
0811 04:41:31.522988 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:31.523004 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:31.523058 (+ 54us) server_negotiation.cc:407] Connection header received
0811 04:41:31.523760 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:31.523780 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:31.524183 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:31.524537 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:31.525387 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:31.526707 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:31.527602 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:31.528093 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:31.528268 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:31.531213 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:31.531218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:31.531232 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:31.531270 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:31.533285 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:31.533809 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:31.533822 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:31.533827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:31.533983 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:31.534357 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:31.534360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:31.534362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:31.535023 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:31.535242 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:31.535548 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 04:41:31.535889 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":182,"threads_started":1}
I20260811 04:41:31.540887 25825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34005 } }
I20260811 04:41:31.541185 25825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:31.541229 25825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5a759d060f456ebf4ef557cbd5dde5, State: Initialized, Role: FOLLOWER
I20260811 04:41:31.541692 25825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34005 } }
I20260811 04:41:31.541842 25825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:31.541900 25825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:31.541987 25825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:31.543025 25825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34005 } }
I20260811 04:41:31.543355 25825 leader_election.cc:302] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b5a759d060f456ebf4ef557cbd5dde5; no voters:
I20260811 04:41:31.543674 25825 leader_election.cc:288] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:31.543831 25830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:31.544118 25830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5a759d060f456ebf4ef557cbd5dde5, State: Running, Role: LEADER
I20260811 04:41:31.544533 25830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34005 } }
I20260811 04:41:31.545217 25825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:31.546482 25832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b5a759d060f456ebf4ef557cbd5dde5. Latest consensus state: current_term: 1 leader_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34005 } } }
I20260811 04:41:31.546491 25831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5a759d060f456ebf4ef557cbd5dde5" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34005 } } }
I20260811 04:41:31.546696 25831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:31.546693 25832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:31.547880 25839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:31.551017 25839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:31.557008 25839 catalog_manager.cc:1347] Generated new cluster ID: 7bb91814bbf7409eb3389225b6d427f9
I20260811 04:41:31.557094 25839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:31.568681 25839 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:31.569823 25839 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:31.581080 25839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b5a759d060f456ebf4ef557cbd5dde5: Generated new TSK 0
I20260811 04:41:31.582132 25839 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:31.624892 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34005
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:31.751251 25852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:31.751559 25852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:31.751655 25852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:31.756386 25852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:31.756480 25852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:31.756548 25852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:31.762609 25852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:31.763765 25852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:31.764849 25852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:31.775442 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:31.775506 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:31.775633 25857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:31.775794 25852 server_base.cc:1034] running on GCE node
I20260811 04:41:31.776322 25852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:31.777045 25852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:31.778271 25852 hybrid_clock.cc:648] HybridClock initialized: now 1786423291778231 us; error 53 us; skew 500 ppm
Aug 11 04:41:31 dist-test-slave-ss4n krb5kdc[25734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:31.781468 25852 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:31.782526 25852 webserver.cc:460] Webserver started at http://127.22.144.129:38199/ using document root <none> and password file <none>
I20260811 04:41:31.783108 25852 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:31.783172 25852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:31.783360 25852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:31.785712 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "1a788b5821654a189047872716992076"
format_stamp: "Formatted at 2026-08-11 04:41:31 on dist-test-slave-ss4n"
server_key: "f6c2764b247d30c05067f2f740d0dca7"
server_key_iv: "aa1c963aef18c20d71fc5e581a045a84"
server_key_version: "encryptionkey@0"
I20260811 04:41:31.786303 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "1a788b5821654a189047872716992076"
format_stamp: "Formatted at 2026-08-11 04:41:31 on dist-test-slave-ss4n"
server_key: "f6c2764b247d30c05067f2f740d0dca7"
server_key_iv: "aa1c963aef18c20d71fc5e581a045a84"
server_key_version: "encryptionkey@0"
I20260811 04:41:31.790916 25852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:41:31.793742 25868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:31.795109 25852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:31.795215 25852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "1a788b5821654a189047872716992076"
format_stamp: "Formatted at 2026-08-11 04:41:31 on dist-test-slave-ss4n"
server_key: "f6c2764b247d30c05067f2f740d0dca7"
server_key_iv: "aa1c963aef18c20d71fc5e581a045a84"
server_key_version: "encryptionkey@0"
I20260811 04:41:31.795311 25852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:31.818499 25852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:31.821938 25852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:31.822108 25852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:31.822697 25852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:31.823586 25852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:31.823635 25852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:31.823676 25852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:31.823691 25852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:31.833741 25852 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:41517
I20260811 04:41:31.833760 25993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:41517 every 8 connection(s)
I20260811 04:41:31.834728 25852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:31.843335 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 25852
I20260811 04:41:31.843483 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:31.843963 23106 external_mini_cluster.cc:1428] Setting key dce85c610e571aea7a4dd8dd6afaf68d
Aug 11 04:41:31 dist-test-slave-ss4n krb5kdc[25734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:31.851636 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:31.836686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:35431 (local address 127.22.144.190:34005)
0811 04:41:31.836873 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:31.836878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:31.837944 (+ 1066us) server_negotiation.cc:407] Connection header received
0811 04:41:31.839252 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:31.839255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:31.839306 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:31.839386 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:31.840757 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:31.841372 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:31.842726 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:31.843022 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:31.843137 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:31.846681 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:31.846687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:31.846694 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:31.846730 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:31.848674 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:31.849447 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:31.849449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:31.849451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:31.849646 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:31.850192 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:31.850196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:31.850197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:31.850415 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:31.850499 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:31.851326 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 04:41:31.851452 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 04:41:31.853129 25999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:31.837194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:34005 (local address 127.22.144.129:35431)
0811 04:41:31.837781 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:31.837826 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:31.839020 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:31.839609 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:31.839616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:31.839989 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:31.840492 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:31.840505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:31.841548 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:31.841551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:31.842512 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:31.842522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:31.843271 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:31.843277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:31.843536 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:31.844348 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:31.844366 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:31.846491 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:31.848822 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:31.848832 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:31.848853 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:31.849325 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:31.849766 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:31.849773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:31.849778 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:31.850027 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:31.850542 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:31.850552 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:31.851012 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 04:41:31.851271 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 04:41:31.851751 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":143,"threads_started":1}
I20260811 04:41:31.854748 25995 heartbeater.cc:347] Connected to a master server at 127.22.144.190:34005
I20260811 04:41:31.855042 25995 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:31.855607 25995 heartbeater.cc:511] Master 127.22.144.190:34005 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:31.857414 25783 ts_manager.cc:194] Registered new tserver with Master: 1a788b5821654a189047872716992076 (127.22.144.129:41517)
I20260811 04:41:31.858645 25783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:35431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:31.882359 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34005
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:32.003815 26002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:32.004101 26002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:32.004148 26002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:32.008217 26002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:32.008283 26002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:32.008342 26002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:32.013895 26002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:32.014959 26002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:32.015973 26002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:32.026415 26008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:32.026351 26010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:32.026468 26007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:32.026638 26002 server_base.cc:1034] running on GCE node
I20260811 04:41:32.027240 26002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:32.027940 26002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:32.029104 26002 hybrid_clock.cc:648] HybridClock initialized: now 1786423292029085 us; error 33 us; skew 500 ppm
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[25734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:32.032110 26002 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:32.033090 26002 webserver.cc:460] Webserver started at http://127.22.144.130:44107/ using document root <none> and password file <none>
I20260811 04:41:32.033737 26002 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:32.033802 26002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:32.034087 26002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:32.036381 26002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "634b0799074949c38e11465595ccc15a"
format_stamp: "Formatted at 2026-08-11 04:41:32 on dist-test-slave-ss4n"
server_key: "c45485ce550652cef96009709e471994"
server_key_iv: "67ab442e980c8f9771eb7d69ea91632d"
server_key_version: "encryptionkey@0"
I20260811 04:41:32.036994 26002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "634b0799074949c38e11465595ccc15a"
format_stamp: "Formatted at 2026-08-11 04:41:32 on dist-test-slave-ss4n"
server_key: "c45485ce550652cef96009709e471994"
server_key_iv: "67ab442e980c8f9771eb7d69ea91632d"
server_key_version: "encryptionkey@0"
I20260811 04:41:32.041004 26002 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:32.043808 26019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:32.045095 26002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:32.045223 26002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "634b0799074949c38e11465595ccc15a"
format_stamp: "Formatted at 2026-08-11 04:41:32 on dist-test-slave-ss4n"
server_key: "c45485ce550652cef96009709e471994"
server_key_iv: "67ab442e980c8f9771eb7d69ea91632d"
server_key_version: "encryptionkey@0"
I20260811 04:41:32.045344 26002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:32.065307 26002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:32.068962 26002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:32.069203 26002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:32.069926 26002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:32.070830 26002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:32.070879 26002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:32.070921 26002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:32.070990 26002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:32.080538 26002 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:45357
I20260811 04:41:32.080577 26145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:45357 every 8 connection(s)
I20260811 04:41:32.081645 26002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:32.091471 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26002
I20260811 04:41:32.091609 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[25734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:32.091917 23106 external_mini_cluster.cc:1428] Setting key ee7eafe47f2c78e4d34a235ab46d33be
I20260811 04:41:32.096838 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:32.083670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:48995 (local address 127.22.144.190:34005)
0811 04:41:32.083813 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:32.083817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:32.084707 (+ 890us) server_negotiation.cc:407] Connection header received
0811 04:41:32.086138 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:32.086141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:32.086200 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:32.086281 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:32.087619 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.088207 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.089459 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.089752 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.089844 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:32.092652 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:32.092655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:32.092657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:32.092708 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:32.094337 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:32.094995 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:32.094998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:32.094999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:32.095090 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:32.095498 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:32.095500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:32.095501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:32.095778 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:32.095896 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:32.096426 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 04:41:32.096646 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:41:32.097693 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:32.084067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:34005 (local address 127.22.144.130:48995)
0811 04:41:32.084502 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:32.084546 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:32.085903 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:32.086462 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:32.086469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:32.086872 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:32.087472 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:32.087488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.088323 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.088326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:32.089325 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:32.089335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.089898 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.089900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:32.090044 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:32.090657 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:32.090673 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:32.092421 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:32.094462 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:32.094471 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:32.094492 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:32.094887 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:32.095185 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:32.095191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:32.095196 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:32.095399 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:32.095908 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:32.095915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:32.096179 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 04:41:32.096341 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 04:41:32.096608 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260811 04:41:32.099041 26146 heartbeater.cc:347] Connected to a master server at 127.22.144.190:34005
I20260811 04:41:32.099434 26146 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:32.100152 26146 heartbeater.cc:511] Master 127.22.144.190:34005 requested a full tablet report, sending...
I20260811 04:41:32.101537 25783 ts_manager.cc:194] Registered new tserver with Master: 634b0799074949c38e11465595ccc15a (127.22.144.130:45357)
I20260811 04:41:32.102396 25783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:48995
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:32.125825 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34005
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:32.244501 26151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:32.244834 26151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:32.244920 26151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:32.249401 26151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:32.249531 26151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:32.249653 26151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:32.255106 26151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:32.256279 26151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:32.257362 26151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:32.268023 26157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:32.268011 26159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:32.268013 26156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:32.268294 26151 server_base.cc:1034] running on GCE node
I20260811 04:41:32.268872 26151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:32.269632 26151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:32.270833 26151 hybrid_clock.cc:648] HybridClock initialized: now 1786423292270791 us; error 59 us; skew 500 ppm
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[25734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:32.274219 26151 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:32.275207 26151 webserver.cc:460] Webserver started at http://127.22.144.131:39389/ using document root <none> and password file <none>
I20260811 04:41:32.275872 26151 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:32.275977 26151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:32.276252 26151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:32.278657 26151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "2b32dd51d0b947a4ae34d80f69828c65"
format_stamp: "Formatted at 2026-08-11 04:41:32 on dist-test-slave-ss4n"
server_key: "06b84f8ac6360f4364aef018d210b93d"
server_key_iv: "0694586e7d5573b24e7dd6a6e348bb9c"
server_key_version: "encryptionkey@0"
I20260811 04:41:32.279386 26151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "2b32dd51d0b947a4ae34d80f69828c65"
format_stamp: "Formatted at 2026-08-11 04:41:32 on dist-test-slave-ss4n"
server_key: "06b84f8ac6360f4364aef018d210b93d"
server_key_iv: "0694586e7d5573b24e7dd6a6e348bb9c"
server_key_version: "encryptionkey@0"
I20260811 04:41:32.283705 26151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:32.286610 26167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:32.288156 26151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:32.288314 26151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "2b32dd51d0b947a4ae34d80f69828c65"
format_stamp: "Formatted at 2026-08-11 04:41:32 on dist-test-slave-ss4n"
server_key: "06b84f8ac6360f4364aef018d210b93d"
server_key_iv: "0694586e7d5573b24e7dd6a6e348bb9c"
server_key_version: "encryptionkey@0"
I20260811 04:41:32.288455 26151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:32.308326 26151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:32.312147 26151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:32.312326 26151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:32.312940 26151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:32.313910 26151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:32.313973 26151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:32.314024 26151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:32.314059 26151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:32.323917 26151 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:46611
I20260811 04:41:32.323942 26289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:46611 every 8 connection(s)
I20260811 04:41:32.324956 26151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:32.334139 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26151
I20260811 04:41:32.334291 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:32.334707 23106 external_mini_cluster.cc:1428] Setting key 2c9265a0ec1c25694e84da32f83a9317
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[25734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:32.340597 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:32.326983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:38463 (local address 127.22.144.190:34005)
0811 04:41:32.327147 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:32.327151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:32.328090 (+ 939us) server_negotiation.cc:407] Connection header received
0811 04:41:32.329623 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:32.329627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:32.329696 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:32.329788 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:32.331150 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.331798 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.333073 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.333337 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.333463 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:32.336734 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:32.336737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:32.336739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:32.336766 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:32.338389 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:32.338902 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:32.338905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:32.338906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:32.338989 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:32.339298 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:32.339300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:32.339301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:32.339489 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:32.339577 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:32.340308 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 04:41:32.340442 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 04:41:32.341444 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:32.327408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:34005 (local address 127.22.144.131:38463)
0811 04:41:32.327917 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:32.327966 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:32.329335 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:32.329971 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:32.329979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:32.330440 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:32.330978 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:32.330993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.331945 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.331949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:32.332932 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:32.332942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:32.333540 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:32.333544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:32.333738 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:32.334445 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:32.334467 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:32.336562 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:32.338522 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:32.338528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:32.338541 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:32.338806 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:32.339074 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:32.339077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:32.339078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:32.339214 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:32.339754 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:32.339760 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:32.340040 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 04:41:32.340228 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 04:41:32.340520 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
I20260811 04:41:32.342772 26290 heartbeater.cc:347] Connected to a master server at 127.22.144.190:34005
I20260811 04:41:32.343152 26290 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:32.343695 26290 heartbeater.cc:511] Master 127.22.144.190:34005 requested a full tablet report, sending...
I20260811 04:41:32.344897 25783 ts_manager.cc:194] Registered new tserver with Master: 2b32dd51d0b947a4ae34d80f69828c65 (127.22.144.131:46611)
I20260811 04:41:32.345752 25783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:38463
I20260811 04:41:32.349365 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 04:41:32.366367 25828 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:32.364665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.22.144.190:34005)
0811 04:41:32.364816 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:32.364821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:32.365005 (+ 184us) server_negotiation.cc:407] Connection header received
0811 04:41:32.365205 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:32.365208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:32.365257 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:32.365347 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:32.366197 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40182: BlockingRecv error: recv got EOF from 127.0.0.1:40182 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:41:32.368345 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 25852
I20260811 04:41:32.377178 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26002
I20260811 04:41:32.386404 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26151
I20260811 04:41:32.395015 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 25749
2026-08-11T04:41:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2389 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[26302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[26302](info): set up 2 sockets
Aug 11 04:41:32 dist-test-slave-ss4n krb5kdc[26302](info): commencing operation
krb5kdc: starting...
W20260811 04:41:33.537688 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:33 dist-test-slave-ss4n krb5kdc[26302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:33.680753 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:43387
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:38719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:33.795739 26317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:33.796069 26317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:33.796146 26317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:33.800240 26317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:33.800336 26317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:33.800376 26317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:33.800436 26317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:33.800472 26317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:33.805785 26317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:38719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:33.806916 26317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:33.808033 26317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:33.816871 26323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:33.816938 26325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:33.816901 26322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:33.817186 26317 server_base.cc:1034] running on GCE node
I20260811 04:41:33.817847 26317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:33.819000 26317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:33.820169 26317 hybrid_clock.cc:648] HybridClock initialized: now 1786423293820148 us; error 32 us; skew 500 ppm
Aug 11 04:41:33 dist-test-slave-ss4n krb5kdc[26302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423293, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:33.823714 26317 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:33.824681 26317 webserver.cc:460] Webserver started at http://127.22.144.190:35461/ using document root <none> and password file <none>
I20260811 04:41:33.825259 26317 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:33.825325 26317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:33.825608 26317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:33.827778 26317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "e38c6034a0cd4e07837935d198d8dc8d"
format_stamp: "Formatted at 2026-08-11 04:41:33 on dist-test-slave-ss4n"
server_key: "3f508009e48c8fb11117aa7451cee787"
server_key_iv: "b1262a1acd6e033086a066f94f10e5a4"
server_key_version: "encryptionkey@0"
I20260811 04:41:33.828384 26317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "e38c6034a0cd4e07837935d198d8dc8d"
format_stamp: "Formatted at 2026-08-11 04:41:33 on dist-test-slave-ss4n"
server_key: "3f508009e48c8fb11117aa7451cee787"
server_key_iv: "b1262a1acd6e033086a066f94f10e5a4"
server_key_version: "encryptionkey@0"
I20260811 04:41:33.832504 26317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:33.835134 26332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:33.836414 26317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:33.836619 26317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "e38c6034a0cd4e07837935d198d8dc8d"
format_stamp: "Formatted at 2026-08-11 04:41:33 on dist-test-slave-ss4n"
server_key: "3f508009e48c8fb11117aa7451cee787"
server_key_iv: "b1262a1acd6e033086a066f94f10e5a4"
server_key_version: "encryptionkey@0"
I20260811 04:41:33.836771 26317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:33.851056 26317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:33.854564 26317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:33.854789 26317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:33.862071 26317 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:43387
I20260811 04:41:33.862368 26390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:43387 every 8 connection(s)
I20260811 04:41:33.863219 26317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:33.866252 26391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:33.868306 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26317
I20260811 04:41:33.868454 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:33.868745 23106 external_mini_cluster.cc:1428] Setting key 157aaa23cea6a59b3b3d805e7be4cdad
I20260811 04:41:33.872927 26391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d: Bootstrap starting.
I20260811 04:41:33.875602 26391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:33.876487 26391 log.cc:824] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:33.878608 26391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d: No bootstrap required, opened a new log
Aug 11 04:41:33 dist-test-slave-ss4n krb5kdc[26302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:33.881732 26391 raft_consensus.cc:348] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38c6034a0cd4e07837935d198d8dc8d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43387 } }
I20260811 04:41:33.881959 26391 raft_consensus.cc:374] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:33.882025 26391 raft_consensus.cc:729] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38c6034a0cd4e07837935d198d8dc8d, State: Initialized, Role: FOLLOWER
I20260811 04:41:33.882489 26391 consensus_queue.cc:260] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38c6034a0cd4e07837935d198d8dc8d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43387 } }
I20260811 04:41:33.882671 26391 raft_consensus.cc:388] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:33.882747 26391 raft_consensus.cc:482] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:33.882856 26391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:33.883919 26391 raft_consensus.cc:504] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38c6034a0cd4e07837935d198d8dc8d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43387 } }
I20260811 04:41:33.884306 26391 leader_election.cc:302] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e38c6034a0cd4e07837935d198d8dc8d; no voters:
I20260811 04:41:33.884683 26391 leader_election.cc:288] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:33.884778 26396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:33.885030 26396 raft_consensus.cc:686] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [term 1 LEADER]: Becoming Leader. State: Replica: e38c6034a0cd4e07837935d198d8dc8d, State: Running, Role: LEADER
I20260811 04:41:33.885337 26396 consensus_queue.cc:237] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38c6034a0cd4e07837935d198d8dc8d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43387 } }
I20260811 04:41:33.885741 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:33.870234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.22.144.190:43387)
0811 04:41:33.870696 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:33.870716 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:33.870762 (+ 46us) server_negotiation.cc:407] Connection header received
0811 04:41:33.871774 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:33.871798 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:33.872229 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:33.872595 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:33.873339 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:33.874840 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:33.875755 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:33.876457 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:33.876639 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:33.879544 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:33.879549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:33.879563 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:33.879591 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:33.882365 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:33.882965 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:33.882974 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:33.882982 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:33.883124 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:33.883436 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:33.883439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:33.883440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:33.883889 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:33.884082 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:33.884360 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 04:41:33.884674 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":115,"threads_started":1}
I20260811 04:41:33.886040 26391 sys_catalog.cc:564] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:33.887107 26397 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e38c6034a0cd4e07837935d198d8dc8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38c6034a0cd4e07837935d198d8dc8d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43387 } } }
I20260811 04:41:33.887233 26397 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:33.887539 26398 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e38c6034a0cd4e07837935d198d8dc8d. Latest consensus state: current_term: 1 leader_uuid: "e38c6034a0cd4e07837935d198d8dc8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38c6034a0cd4e07837935d198d8dc8d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43387 } } }
I20260811 04:41:33.887627 26398 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:33.889133 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 04:41:33.891099 26412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:33.891181 26412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:33.891268 26406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:33.892161 26406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:33.897578 26406 catalog_manager.cc:1347] Generated new cluster ID: fd79f9a28ba44e55a4a719ecfadf1bc6
I20260811 04:41:33.897651 26406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:33.909647 26406 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:33.910750 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:33.904790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.22.144.190:43387)
0811 04:41:33.904911 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:33.904916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:33.905200 (+ 284us) server_negotiation.cc:407] Connection header received
0811 04:41:33.905328 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:33.905331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:33.905390 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:33.905529 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:33.906103 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:33.906928 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:33.908050 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:33.908427 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:33.908537 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:33.909688 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:33.909692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:33.909694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:33.910236 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:33.910268 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:33.910286 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:33.910337 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:33.910522 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 04:41:33.910589 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 04:41:33.910900 26406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:33.927505 26406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e38c6034a0cd4e07837935d198d8dc8d: Generated new TSK 0
I20260811 04:41:33.928318 26406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:41:33.934653 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26317
2026-08-11T04:41:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1540 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:34 dist-test-slave-ss4n krb5kdc[26428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:34 dist-test-slave-ss4n krb5kdc[26428](info): set up 2 sockets
Aug 11 04:41:34 dist-test-slave-ss4n krb5kdc[26428](info): commencing operation
krb5kdc: starting...
W20260811 04:41:35.052033 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:35.193327 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44287
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:44287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:35.307564 26442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:35.307909 26442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:35.307992 26442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:35.312085 26442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:35.312201 26442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:35.312242 26442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:35.312280 26442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:35.312316 26442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:35.317787 26442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:44287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:35.318984 26442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:35.320086 26442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:35.330024 26448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:35.330130 26452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:35.330024 26449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:35.330461 26442 server_base.cc:1034] running on GCE node
I20260811 04:41:35.331095 26442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:35.332417 26442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:35.333621 26442 hybrid_clock.cc:648] HybridClock initialized: now 1786423295333607 us; error 33 us; skew 500 ppm
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:35.341764 26442 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:35.342820 26442 webserver.cc:460] Webserver started at http://127.22.144.190:33357/ using document root <none> and password file <none>
I20260811 04:41:35.343428 26442 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:35.343508 26442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:35.343730 26442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:35.346009 26442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "3ffddac22bbd23fd6b4182a8577949f0"
server_key_iv: "ef3675e13dfe9f1de9412d6b1f8f82a9"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.346619 26442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "3ffddac22bbd23fd6b4182a8577949f0"
server_key_iv: "ef3675e13dfe9f1de9412d6b1f8f82a9"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.350828 26442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:41:35.353608 26460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.355019 26442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:41:35.355180 26442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "3ffddac22bbd23fd6b4182a8577949f0"
server_key_iv: "ef3675e13dfe9f1de9412d6b1f8f82a9"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.355309 26442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:35.372916 26442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:35.376195 26442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:35.376421 26442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:35.383957 26442 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:44287
I20260811 04:41:35.384277 26514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:44287 every 8 connection(s)
I20260811 04:41:35.385004 26442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:35.387851 26515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:35.391362 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26442
I20260811 04:41:35.391525 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:35.391808 23106 external_mini_cluster.cc:1428] Setting key 15d7f0e8019709d7416ba8827d5363da
I20260811 04:41:35.394155 26515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7: Bootstrap starting.
I20260811 04:41:35.396682 26515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:35.397436 26515 log.cc:824] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:35.399735 26515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7: No bootstrap required, opened a new log
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:35.404814 26515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44287 } }
I20260811 04:41:35.405092 26515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:35.405171 26515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2b68f1bb5b4f6f9e05e589015c05d7, State: Initialized, Role: FOLLOWER
I20260811 04:41:35.405740 26515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44287 } }
I20260811 04:41:35.405905 26515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:35.405977 26515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:35.406073 26515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:35.407311 26515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44287 } }
I20260811 04:41:35.407683 26515 leader_election.cc:302] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e2b68f1bb5b4f6f9e05e589015c05d7; no voters:
I20260811 04:41:35.408033 26515 leader_election.cc:288] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:35.408071 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:35.393562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.22.144.190:44287)
0811 04:41:35.394090 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:35.394107 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:35.394145 (+ 38us) server_negotiation.cc:407] Connection header received
0811 04:41:35.394888 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:35.394907 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:35.395243 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:35.395613 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:35.396452 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.397770 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.398914 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.399393 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.399521 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:35.402469 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:35.402475 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:35.402493 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:35.402526 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:35.404593 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:35.405054 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:35.405062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:35.405068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:35.405255 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:35.405602 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:35.405605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:35.405607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:35.406169 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:35.406384 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:35.406714 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 04:41:35.407046 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":142,"threads_started":1}
I20260811 04:41:35.408179 26521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:35.408408 26521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4e2b68f1bb5b4f6f9e05e589015c05d7, State: Running, Role: LEADER
I20260811 04:41:35.408753 26521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44287 } }
I20260811 04:41:35.409341 26515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:35.409984 26521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e2b68f1bb5b4f6f9e05e589015c05d7. Latest consensus state: current_term: 1 leader_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44287 } } }
I20260811 04:41:35.410228 26521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:35.410837 26527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:35.411134 26522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2b68f1bb5b4f6f9e05e589015c05d7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44287 } } }
I20260811 04:41:35.411269 26522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:35.414165 26527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:35.419955 26527 catalog_manager.cc:1347] Generated new cluster ID: 540c67c6cba64038a0fb37cb0fa1e6bf
I20260811 04:41:35.420044 26527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:35.429503 26527 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:35.430716 26527 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:35.439903 26527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e2b68f1bb5b4f6f9e05e589015c05d7: Generated new TSK 0
I20260811 04:41:35.440737 26527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:35.481006 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44287
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:35.597543 26542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:35.597802 26542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:35.597846 26542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:35.601787 26542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:35.601836 26542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:35.601893 26542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:35.607076 26542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:35.608234 26542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:35.609382 26542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:35.619773 26551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:35.619804 26548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:35.619836 26549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:35.619977 26542 server_base.cc:1034] running on GCE node
I20260811 04:41:35.620569 26542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:35.621168 26542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:35.622360 26542 hybrid_clock.cc:648] HybridClock initialized: now 1786423295622321 us; error 49 us; skew 500 ppm
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:35.625782 26542 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:35.626792 26542 webserver.cc:460] Webserver started at http://127.22.144.129:35879/ using document root <none> and password file <none>
I20260811 04:41:35.627363 26542 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:35.627429 26542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:35.627688 26542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:35.630023 26542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "c5fa69b94bc34657a378a01623de912a"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "98e2f6dbca9bc5dda8d6afb0b363d51d"
server_key_iv: "23b19a70697cce56d7200ca5ba93716e"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.630703 26542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "c5fa69b94bc34657a378a01623de912a"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "98e2f6dbca9bc5dda8d6afb0b363d51d"
server_key_iv: "23b19a70697cce56d7200ca5ba93716e"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.634913 26542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:35.637801 26558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.639211 26542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:41:35.639375 26542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "c5fa69b94bc34657a378a01623de912a"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "98e2f6dbca9bc5dda8d6afb0b363d51d"
server_key_iv: "23b19a70697cce56d7200ca5ba93716e"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.639520 26542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:35.652104 26542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:35.655375 26542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:35.655582 26542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:35.656178 26542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:35.657014 26542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:35.657084 26542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.657156 26542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:35.657208 26542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.666662 26542 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:36125
I20260811 04:41:35.666690 26682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:36125 every 8 connection(s)
I20260811 04:41:35.667569 26542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:35.668264 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26542
I20260811 04:41:35.668401 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:35.668982 23106 external_mini_cluster.cc:1428] Setting key b2c8dcf1e0b1eff782fc859a9949ff37
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:35.685319 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:35.669854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:51057 (local address 127.22.144.190:44287)
0811 04:41:35.670002 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:35.670008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:35.671226 (+ 1218us) server_negotiation.cc:407] Connection header received
0811 04:41:35.673530 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:35.673534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:35.673596 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:35.673687 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:35.675054 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.675676 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.677087 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.677363 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.677532 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:35.680856 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:35.680863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:35.680868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:35.680906 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:35.682625 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:35.683267 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:35.683270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:35.683271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:35.683356 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:35.683795 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:35.683802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:35.683808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:35.684148 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:35.684249 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:35.685049 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 04:41:35.685180 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:41:35.686594 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:35.670366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44287 (local address 127.22.144.129:51057)
0811 04:41:35.671028 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:35.671093 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:35.673249 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:35.673886 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:35.673894 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:35.674319 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:35.674869 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:35.674884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.675829 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.675833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:35.676929 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:35.676950 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.677579 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.677584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:35.677787 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:35.678514 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:35.678532 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:35.680581 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:35.682777 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:35.682787 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:35.682805 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:35.683151 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:35.683459 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:35.683462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:35.683465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:35.683642 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:35.684275 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:35.684285 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:35.684674 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 04:41:35.684961 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 04:41:35.685473 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":181,"threads_started":1}
I20260811 04:41:35.688045 26683 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44287
I20260811 04:41:35.688336 26683 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:35.689049 26683 heartbeater.cc:511] Master 127.22.144.190:44287 requested a full tablet report, sending...
I20260811 04:41:35.691044 26478 ts_manager.cc:194] Registered new tserver with Master: c5fa69b94bc34657a378a01623de912a (127.22.144.129:36125)
I20260811 04:41:35.692344 26478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:51057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:35.704502 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44287
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:35.824136 26688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:35.824465 26688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:35.824571 26688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:35.829326 26688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:35.829432 26688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:35.829576 26688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:35.836150 26688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:35.837392 26688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:35.838585 26688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:41:35.849934 26688 server_base.cc:1034] running on GCE node
W20260811 04:41:35.849956 26693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:35.849888 26694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:35.850241 26696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:35.850773 26688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:35.851511 26688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:35.852737 26688 hybrid_clock.cc:648] HybridClock initialized: now 1786423295852708 us; error 50 us; skew 500 ppm
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:35.855954 26688 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:35.856972 26688 webserver.cc:460] Webserver started at http://127.22.144.130:32841/ using document root <none> and password file <none>
I20260811 04:41:35.857765 26688 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:35.857878 26688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:35.858163 26688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:35.860651 26688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "19209acc6b3b464e89b24c3cd9d8d218"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "d61b55acb12318fbe6ba36a86bc00969"
server_key_iv: "3c387d4493f327c43af860449876a5ad"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.861284 26688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "19209acc6b3b464e89b24c3cd9d8d218"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "d61b55acb12318fbe6ba36a86bc00969"
server_key_iv: "3c387d4493f327c43af860449876a5ad"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.865715 26688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:41:35.868510 26703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.870149 26688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:35.870314 26688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "19209acc6b3b464e89b24c3cd9d8d218"
format_stamp: "Formatted at 2026-08-11 04:41:35 on dist-test-slave-ss4n"
server_key: "d61b55acb12318fbe6ba36a86bc00969"
server_key_iv: "3c387d4493f327c43af860449876a5ad"
server_key_version: "encryptionkey@0"
I20260811 04:41:35.870450 26688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:35.889823 26688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:35.893810 26688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:35.894053 26688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:35.894742 26688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:35.895720 26688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:35.895799 26688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.895880 26688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:35.895937 26688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:35.906286 26688 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:46099
I20260811 04:41:35.906307 26823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:46099 every 8 connection(s)
I20260811 04:41:35.907316 26688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:35.913017 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26688
I20260811 04:41:35.913154 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:35.913457 23106 external_mini_cluster.cc:1428] Setting key fc317f869b0932d1cc901c8241ea2343
Aug 11 04:41:35 dist-test-slave-ss4n krb5kdc[26428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423295, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:35.926321 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:35.909406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:44295 (local address 127.22.144.190:44287)
0811 04:41:35.909623 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:35.909627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:35.910638 (+ 1011us) server_negotiation.cc:407] Connection header received
0811 04:41:35.912313 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:35.912316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:35.912367 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:35.912474 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:35.914288 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.915128 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.917382 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.917836 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.917971 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:35.921635 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:35.921639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:35.921642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:35.921672 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:35.923762 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:35.924456 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:35.924459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:35.924461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:35.924558 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:35.924987 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:35.924991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:35.924999 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:35.925211 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:35.925320 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:35.925978 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 04:41:35.926129 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:41:35.927731 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:35.909898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44287 (local address 127.22.144.130:44295)
0811 04:41:35.910459 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:35.910508 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:35.912018 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:35.912668 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:35.912676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:35.913204 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:35.914091 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:35.914120 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.915315 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.915321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:35.917211 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:35.917229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:35.918129 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:35.918137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:35.918399 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:35.919267 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:35.919327 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:35.921374 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:35.923916 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:35.923923 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:35.923943 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:35.924350 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:35.924660 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:35.924663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:35.924665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:35.924875 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:35.925336 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:35.925345 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:35.925709 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 04:41:35.925915 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 04:41:35.926816 (+ 901us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":151,"threads_started":1}
I20260811 04:41:35.929184 26824 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44287
I20260811 04:41:35.929579 26824 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:35.930229 26824 heartbeater.cc:511] Master 127.22.144.190:44287 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:41:35.931672 26478 ts_manager.cc:194] Registered new tserver with Master: 19209acc6b3b464e89b24c3cd9d8d218 (127.22.144.130:46099)
I20260811 04:41:35.932318 26478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:44295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:35.949854 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44287
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:36.069602 26830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:36.069866 26830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:36.069909 26830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:36.073887 26830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:36.073943 26830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:36.074000 26830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:36.079285 26830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:44287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:36.080302 26830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:36.081399 26830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:36.091410 26836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:36.091495 26830 server_base.cc:1034] running on GCE node
W20260811 04:41:36.091440 26835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:36.091410 26838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:36.092163 26830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:36.092849 26830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:36.094027 26830 hybrid_clock.cc:648] HybridClock initialized: now 1786423296094005 us; error 35 us; skew 500 ppm
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:36.097134 26830 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:36.098081 26830 webserver.cc:460] Webserver started at http://127.22.144.131:36017/ using document root <none> and password file <none>
I20260811 04:41:36.098675 26830 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:36.098737 26830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:36.098906 26830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:36.100956 26830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "9c1fe16fcca04e1aae72793ad5bc8ebd"
format_stamp: "Formatted at 2026-08-11 04:41:36 on dist-test-slave-ss4n"
server_key: "260c2cb92b5574e6efb01ae7a8b6630c"
server_key_iv: "7b7ca82c895eefe4f51f7a2a70b99e84"
server_key_version: "encryptionkey@0"
I20260811 04:41:36.101586 26830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "9c1fe16fcca04e1aae72793ad5bc8ebd"
format_stamp: "Formatted at 2026-08-11 04:41:36 on dist-test-slave-ss4n"
server_key: "260c2cb92b5574e6efb01ae7a8b6630c"
server_key_iv: "7b7ca82c895eefe4f51f7a2a70b99e84"
server_key_version: "encryptionkey@0"
I20260811 04:41:36.105298 26830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:36.108026 26846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:36.109311 26830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:36.109447 26830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "9c1fe16fcca04e1aae72793ad5bc8ebd"
format_stamp: "Formatted at 2026-08-11 04:41:36 on dist-test-slave-ss4n"
server_key: "260c2cb92b5574e6efb01ae7a8b6630c"
server_key_iv: "7b7ca82c895eefe4f51f7a2a70b99e84"
server_key_version: "encryptionkey@0"
I20260811 04:41:36.109616 26830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:36.124771 26830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:36.128727 26830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:36.128968 26830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:36.129832 26830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:36.130702 26830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:36.130753 26830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:36.130798 26830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:36.130815 26830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:36.141739 26830 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:46523
I20260811 04:41:36.141809 26965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:46523 every 8 connection(s)
I20260811 04:41:36.142865 26830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:36.148375 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26830
I20260811 04:41:36.148501 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:36.148866 23106 external_mini_cluster.cc:1428] Setting key 0c260693017f5eccc59a30cd829c4926
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:36.158981 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:36.145091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:59461 (local address 127.22.144.190:44287)
0811 04:41:36.145244 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:36.145248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:36.146197 (+ 949us) server_negotiation.cc:407] Connection header received
0811 04:41:36.147674 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:36.147677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:36.147736 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:36.147827 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:36.149255 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:36.150020 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:36.151481 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:36.151724 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:36.151833 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:36.154638 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:36.154642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:36.154647 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:36.154683 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:36.156361 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:36.157073 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:36.157086 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:36.157088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:36.157191 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:36.157660 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:36.157668 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:36.157670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:36.157893 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:36.158021 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:36.158625 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 04:41:36.158765 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:41:36.159878 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:36.145526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:44287 (local address 127.22.144.131:59461)
0811 04:41:36.146010 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:36.146065 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:36.147417 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:36.147991 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:36.147997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:36.148428 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:36.149097 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:36.149114 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:36.150148 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:36.150153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:36.151340 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:36.151351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:36.151889 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:36.151893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:36.152079 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:36.152673 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:36.152690 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:36.154437 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:36.156608 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:36.156615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:36.156632 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:36.156949 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:36.157305 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:36.157309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:36.157311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:36.157530 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:36.158020 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:36.158028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:36.158351 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 04:41:36.158560 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 04:41:36.158924 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":114,"threads_started":1}
I20260811 04:41:36.161192 26966 heartbeater.cc:347] Connected to a master server at 127.22.144.190:44287
I20260811 04:41:36.161432 26966 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:36.162101 26966 heartbeater.cc:511] Master 127.22.144.190:44287 requested a full tablet report, sending...
I20260811 04:41:36.163589 26477 ts_manager.cc:194] Registered new tserver with Master: 9c1fe16fcca04e1aae72793ad5bc8ebd (127.22.144.131:46523)
I20260811 04:41:36.164276 26477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:59461
I20260811 04:41:36.175743 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423296, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423296, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:36.206292 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:36.196255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.22.144.190:44287)
0811 04:41:36.196467 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:36.196472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:36.196698 (+ 226us) server_negotiation.cc:407] Connection header received
0811 04:41:36.196978 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:36.196985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:36.197058 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:36.197215 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:36.198128 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:36.198718 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:36.199706 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:36.199957 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:36.200085 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:36.202821 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:36.202824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:36.202826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:36.202853 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:36.204245 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:36.204763 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:36.204767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:36.204769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:36.204864 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:36.205210 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:36.205215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:36.205217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:36.205531 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:36.205693 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:36.205963 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 04:41:36.206105 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260811 04:41:36.206760 26477 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45208
I20260811 04:41:36.207834 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26542
I20260811 04:41:36.221436 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26688
I20260811 04:41:36.229127 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26830
I20260811 04:41:36.238112 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26442
2026-08-11T04:41:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2301 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26979](info): set up 2 sockets
Aug 11 04:41:36 dist-test-slave-ss4n krb5kdc[26979](info): commencing operation
krb5kdc: starting...
W20260811 04:41:37.430094 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:37 dist-test-slave-ss4n krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:37.574064 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42053
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:37.695257 26992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:37.695684 26992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:37.695770 26992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:37.699898 26992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:37.700016 26992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:37.700055 26992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:37.700089 26992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:37.700129 26992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:37.705699 26992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:37.706864 26992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:37.707976 26992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:37.717419 26997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:37.717545 26992 server_base.cc:1034] running on GCE node
W20260811 04:41:37.717433 27002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:37.717420 26998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:37.718300 26992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:37.719422 26992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:37.720618 26992 hybrid_clock.cc:648] HybridClock initialized: now 1786423297720554 us; error 48 us; skew 500 ppm
Aug 11 04:41:37 dist-test-slave-ss4n krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423297, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:37.723626 26992 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:37.724514 26992 webserver.cc:460] Webserver started at http://127.22.144.190:45235/ using document root <none> and password file <none>
I20260811 04:41:37.725095 26992 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:37.725158 26992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:37.725392 26992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:37.727523 26992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "8298038844be4a08a277987847568373"
format_stamp: "Formatted at 2026-08-11 04:41:37 on dist-test-slave-ss4n"
server_key: "94b24729ac947da4b639c4c57b63ac39"
server_key_iv: "bd28443d1f2684aa65aa6e5c03a5af3b"
server_key_version: "encryptionkey@0"
I20260811 04:41:37.728114 26992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "8298038844be4a08a277987847568373"
format_stamp: "Formatted at 2026-08-11 04:41:37 on dist-test-slave-ss4n"
server_key: "94b24729ac947da4b639c4c57b63ac39"
server_key_iv: "bd28443d1f2684aa65aa6e5c03a5af3b"
server_key_version: "encryptionkey@0"
I20260811 04:41:37.732065 26992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 04:41:37.734896 27009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:37.736178 26992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:37.736317 26992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "8298038844be4a08a277987847568373"
format_stamp: "Formatted at 2026-08-11 04:41:37 on dist-test-slave-ss4n"
server_key: "94b24729ac947da4b639c4c57b63ac39"
server_key_iv: "bd28443d1f2684aa65aa6e5c03a5af3b"
server_key_version: "encryptionkey@0"
I20260811 04:41:37.736430 26992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:37.752754 26992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:37.756378 26992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:37.756623 26992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:37.764578 26992 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:42053
I20260811 04:41:37.764597 27063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:42053 every 8 connection(s)
I20260811 04:41:37.765652 26992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:37.768627 27064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:37.772053 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 26992
I20260811 04:41:37.772168 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:37.772390 23106 external_mini_cluster.cc:1428] Setting key be986d0386be578e9c13eeef51498613
I20260811 04:41:37.774946 27064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373: Bootstrap starting.
I20260811 04:41:37.777235 27064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:37.778079 27064 log.cc:824] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:37.780009 27064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373: No bootstrap required, opened a new log
Aug 11 04:41:37 dist-test-slave-ss4n krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:37.783651 27064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8298038844be4a08a277987847568373" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42053 } }
I20260811 04:41:37.783879 27064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:37.783995 27064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8298038844be4a08a277987847568373, State: Initialized, Role: FOLLOWER
I20260811 04:41:37.784478 27064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8298038844be4a08a277987847568373" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42053 } }
I20260811 04:41:37.784608 27064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:37.784699 27064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:37.784824 27064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:37.786123 27064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8298038844be4a08a277987847568373" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42053 } }
I20260811 04:41:37.786485 27064 leader_election.cc:302] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8298038844be4a08a277987847568373; no voters:
I20260811 04:41:37.786861 27064 leader_election.cc:288] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:37.786911 27069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:37.787117 27069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [term 1 LEADER]: Becoming Leader. State: Replica: 8298038844be4a08a277987847568373, State: Running, Role: LEADER
I20260811 04:41:37.787482 27069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8298038844be4a08a277987847568373" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42053 } }
I20260811 04:41:37.788231 27064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:37.788740 27069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8298038844be4a08a277987847568373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8298038844be4a08a277987847568373" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42053 } } }
I20260811 04:41:37.789187 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:37.773912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.22.144.190:42053)
0811 04:41:37.774398 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:37.774414 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:37.774460 (+ 46us) server_negotiation.cc:407] Connection header received
0811 04:41:37.775122 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:37.775140 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:37.775477 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:37.775829 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:37.776665 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:37.777817 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:37.778759 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:37.779242 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:37.779427 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:37.782332 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:37.782337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:37.782351 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:37.782378 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:37.785097 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:37.785624 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:37.785629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:37.785634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:37.785747 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:37.786090 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:37.786093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:37.786094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:37.786558 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:37.786802 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:37.787588 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 04:41:37.787894 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":149,"threads_started":1}
I20260811 04:41:37.789924 27069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:37.789368 27070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8298038844be4a08a277987847568373. Latest consensus state: current_term: 1 leader_uuid: "8298038844be4a08a277987847568373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8298038844be4a08a277987847568373" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42053 } } }
I20260811 04:41:37.790217 27070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373 [sys.catalog]: This master's current role is: LEADER
W20260811 04:41:37.793380 27085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:37.793463 27085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:37.793545 27080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:37.794243 27080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:37.800298 27080 catalog_manager.cc:1347] Generated new cluster ID: aeb44a121c504f9abba19033a78b87db
I20260811 04:41:37.800421 27080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:37.808045 27080 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:37.809120 27080 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:37.818638 27080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8298038844be4a08a277987847568373: Generated new TSK 0
I20260811 04:41:37.819507 27080 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:37.860846 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42053
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:37.984408 27091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:37.984683 27091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:37.984725 27091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:37.989034 27091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:37.989096 27091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:37.989157 27091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:37.994661 27091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:37.995834 27091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:37.996958 27091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:38.007498 27096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:38.007684 27091 server_base.cc:1034] running on GCE node
W20260811 04:41:38.007498 27099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:38.007570 27097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:38.008211 27091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:38.009142 27091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:38.010391 27091 hybrid_clock.cc:648] HybridClock initialized: now 1786423298010381 us; error 48 us; skew 500 ppm
Aug 11 04:41:38 dist-test-slave-ss4n krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:38.013402 27091 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:38.014340 27091 webserver.cc:460] Webserver started at http://127.22.144.129:40069/ using document root <none> and password file <none>
I20260811 04:41:38.014884 27091 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:38.014946 27091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:38.015130 27091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:38.017215 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "bcaac9ad06804431b04cf1f68841e097"
format_stamp: "Formatted at 2026-08-11 04:41:38 on dist-test-slave-ss4n"
server_key: "c190920227635b6169c2133e089563ae"
server_key_iv: "48f9645d4316d0334cbb123e40be2648"
server_key_version: "encryptionkey@0"
I20260811 04:41:38.017860 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "bcaac9ad06804431b04cf1f68841e097"
format_stamp: "Formatted at 2026-08-11 04:41:38 on dist-test-slave-ss4n"
server_key: "c190920227635b6169c2133e089563ae"
server_key_iv: "48f9645d4316d0334cbb123e40be2648"
server_key_version: "encryptionkey@0"
I20260811 04:41:38.021927 27091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:38.024529 27107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:38.025904 27091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:38.026005 27091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "bcaac9ad06804431b04cf1f68841e097"
format_stamp: "Formatted at 2026-08-11 04:41:38 on dist-test-slave-ss4n"
server_key: "c190920227635b6169c2133e089563ae"
server_key_iv: "48f9645d4316d0334cbb123e40be2648"
server_key_version: "encryptionkey@0"
I20260811 04:41:38.026099 27091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:38.037258 27091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:38.040467 27091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:38.040688 27091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:38.041388 27091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:38.042312 27091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:38.042367 27091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:38.042445 27091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:38.042476 27091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:38.052233 27091 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:34675
I20260811 04:41:38.052284 27231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:34675 every 8 connection(s)
I20260811 04:41:38.053153 27091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:38.059595 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27091
I20260811 04:41:38.059772 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:38.060089 23106 external_mini_cluster.cc:1428] Setting key ebbab8280d49714b43e8391422bf4984
Aug 11 04:41:38 dist-test-slave-ss4n krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:38.068454 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:38.055023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:57503 (local address 127.22.144.190:42053)
0811 04:41:38.055162 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:38.055166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:38.056029 (+ 863us) server_negotiation.cc:407] Connection header received
0811 04:41:38.057516 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:38.057520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:38.057625 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:38.057729 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:38.058998 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.059547 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.060948 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.061206 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.061326 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:38.064347 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:38.064350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:38.064352 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:38.064378 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:38.065994 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:38.066595 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:38.066601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:38.066606 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:38.066722 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:38.067086 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:38.067089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:38.067091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:38.067381 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:38.067507 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:38.068044 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 04:41:38.068204 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:41:38.069096 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:38.055401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42053 (local address 127.22.144.129:57503)
0811 04:41:38.055860 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:38.055903 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:38.057267 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:38.057912 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:38.057918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:38.058334 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:38.058835 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:38.058850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.059699 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.059702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:38.060789 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:38.060799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.061371 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.061374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:38.061628 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:38.062309 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:38.062325 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:38.064159 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:38.066125 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:38.066131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:38.066148 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:38.066429 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:38.066833 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:38.066835 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:38.066836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:38.066983 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:38.067500 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:38.067506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:38.067780 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 04:41:38.067946 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 04:41:38.068213 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260811 04:41:38.070427 27232 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42053
I20260811 04:41:38.070698 27232 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:38.071244 27232 heartbeater.cc:511] Master 127.22.144.190:42053 requested a full tablet report, sending...
I20260811 04:41:38.072861 27026 ts_manager.cc:194] Registered new tserver with Master: bcaac9ad06804431b04cf1f68841e097 (127.22.144.129:34675)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:38.074323 27026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:57503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:38.094336 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42053
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:38.227785 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:38.228132 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:38.228215 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:38.232690 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:38.232798 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:38.232898 27237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:38.238621 27237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:38.239929 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:38.241111 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:38.252134 27245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:38.252130 27242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:38.252494 27237 server_base.cc:1034] running on GCE node
W20260811 04:41:38.252115 27243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:38.252977 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:38.253825 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:38.255088 27237 hybrid_clock.cc:648] HybridClock initialized: now 1786423298255047 us; error 57 us; skew 500 ppm
Aug 11 04:41:38 dist-test-slave-ss4n krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:38.258486 27237 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:38.259527 27237 webserver.cc:460] Webserver started at http://127.22.144.130:45659/ using document root <none> and password file <none>
I20260811 04:41:38.260071 27237 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:38.260131 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:38.260299 27237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:38.262593 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "7b1cb393c1ed43668e77109fc80af9e6"
format_stamp: "Formatted at 2026-08-11 04:41:38 on dist-test-slave-ss4n"
server_key: "f6409028a22d26cf86963af39257faa4"
server_key_iv: "552529c3098e8eae84eb73eccd56200f"
server_key_version: "encryptionkey@0"
I20260811 04:41:38.263206 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "7b1cb393c1ed43668e77109fc80af9e6"
format_stamp: "Formatted at 2026-08-11 04:41:38 on dist-test-slave-ss4n"
server_key: "f6409028a22d26cf86963af39257faa4"
server_key_iv: "552529c3098e8eae84eb73eccd56200f"
server_key_version: "encryptionkey@0"
I20260811 04:41:38.267532 27237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:38.270664 27252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:38.272214 27237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:41:38.272349 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "7b1cb393c1ed43668e77109fc80af9e6"
format_stamp: "Formatted at 2026-08-11 04:41:38 on dist-test-slave-ss4n"
server_key: "f6409028a22d26cf86963af39257faa4"
server_key_iv: "552529c3098e8eae84eb73eccd56200f"
server_key_version: "encryptionkey@0"
I20260811 04:41:38.272516 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:38.303483 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:38.307040 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:38.307261 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:38.308004 27237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:38.308992 27237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:38.309053 27237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:38.309129 27237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:38.309168 27237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:38.319336 27237 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:33629
I20260811 04:41:38.319356 27375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:33629 every 8 connection(s)
I20260811 04:41:38.320312 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:38.323148 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27237
I20260811 04:41:38.323285 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:38.323611 23106 external_mini_cluster.cc:1428] Setting key dc6aba0288070ce5acbc10d9b87dd08e
Aug 11 04:41:38 dist-test-slave-ss4n krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423298, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:41:38.337944 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:38.322379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:32965 (local address 127.22.144.190:42053)
0811 04:41:38.322546 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:38.322550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:38.323803 (+ 1253us) server_negotiation.cc:407] Connection header received
0811 04:41:38.325699 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:38.325703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:38.325760 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:38.325836 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:38.327100 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.327636 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.328860 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.329285 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.329406 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:38.332911 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:38.332917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:38.332920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:38.332957 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:38.335070 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:38.335855 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:38.335859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:38.335860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:38.335979 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:38.336538 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:38.336541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:38.336542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:38.336766 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:38.336857 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:38.337679 (+ 822us) server_negotiation.cc:299] Negotiation successful
0811 04:41:38.337814 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:41:38.338889 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:38.322932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42053 (local address 127.22.144.130:32965)
0811 04:41:38.323601 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:38.323673 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:38.325408 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:38.326028 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:38.326036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:38.326411 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:38.326936 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:38.326950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.327778 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.327782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:38.328691 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:38.328701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:38.329662 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:38.329667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:38.329844 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:38.330678 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:38.330705 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:38.332666 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:38.335225 (+ 2559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:38.335236 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:38.335259 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:38.335718 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:38.336162 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:38.336169 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:38.336175 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:38.336395 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:38.336893 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:38.336903 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:38.337293 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 04:41:38.337577 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 04:41:38.337966 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":156,"threads_started":1}
I20260811 04:41:38.340421 27376 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42053
I20260811 04:41:38.340746 27376 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:38.341383 27376 heartbeater.cc:511] Master 127.22.144.190:42053 requested a full tablet report, sending...
I20260811 04:41:38.342777 27026 ts_manager.cc:194] Registered new tserver with Master: 7b1cb393c1ed43668e77109fc80af9e6 (127.22.144.130:33629)
I20260811 04:41:38.343464 27026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:32965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:41:39.076934 27232 heartbeater.cc:503] Master 127.22.144.190:42053 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
W20260811 04:41:39.174556 23106 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.144.131: real 0.849s user 0.004s sys 0.000s
I20260811 04:41:39.175208 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42053
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:39.300925 27382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:39.301241 27382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:39.301287 27382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:39.305851 27382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:39.305934 27382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:39.305999 27382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:39.311787 27382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:39.312994 27382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:39.314134 27382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:39.324589 27388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:39.324595 27387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:39.324620 27390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:39.325151 27382 server_base.cc:1034] running on GCE node
I20260811 04:41:39.325706 27382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:39.326400 27382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:39.329710 27382 hybrid_clock.cc:648] HybridClock initialized: now 1786423299329674 us; error 45 us; skew 500 ppm
Aug 11 04:41:39 dist-test-slave-ss4n krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423299, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:39.332990 27382 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:39.334118 27382 webserver.cc:460] Webserver started at http://127.22.144.131:46633/ using document root <none> and password file <none>
I20260811 04:41:39.334750 27382 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:39.334822 27382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:39.335062 27382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:39.337288 27382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "75065f5104524927b8b91d93ff5f7fd5"
format_stamp: "Formatted at 2026-08-11 04:41:39 on dist-test-slave-ss4n"
server_key: "d98a30ef0a81648ff6cd319d872abd42"
server_key_iv: "0697ab18e1eff446d488c9fb2178c321"
server_key_version: "encryptionkey@0"
I20260811 04:41:39.338001 27382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "75065f5104524927b8b91d93ff5f7fd5"
format_stamp: "Formatted at 2026-08-11 04:41:39 on dist-test-slave-ss4n"
server_key: "d98a30ef0a81648ff6cd319d872abd42"
server_key_iv: "0697ab18e1eff446d488c9fb2178c321"
server_key_version: "encryptionkey@0"
I20260811 04:41:39.342530 27382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:39.345377 27397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:39.345780 27376 heartbeater.cc:503] Master 127.22.144.190:42053 was elected leader, sending a full tablet report...
I20260811 04:41:39.347184 27382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:39.347316 27382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "75065f5104524927b8b91d93ff5f7fd5"
format_stamp: "Formatted at 2026-08-11 04:41:39 on dist-test-slave-ss4n"
server_key: "d98a30ef0a81648ff6cd319d872abd42"
server_key_iv: "0697ab18e1eff446d488c9fb2178c321"
server_key_version: "encryptionkey@0"
I20260811 04:41:39.347443 27382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:39.357203 27382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:39.360787 27382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:39.361038 27382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:39.362457 27382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:39.363399 27382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:39.363449 27382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:39.363490 27382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:39.363507 27382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:39.373225 27382 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:37429
I20260811 04:41:39.373247 27521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:37429 every 8 connection(s)
I20260811 04:41:39.374270 27382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:39.383605 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27382
I20260811 04:41:39.383780 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:39.384174 23106 external_mini_cluster.cc:1428] Setting key f3a01ac520ab4ea5dce71bb7ad009768
Aug 11 04:41:39 dist-test-slave-ss4n krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423299, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:39.390575 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:39.376335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:42555 (local address 127.22.144.190:42053)
0811 04:41:39.376598 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:39.376604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:39.377562 (+ 958us) server_negotiation.cc:407] Connection header received
0811 04:41:39.379170 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:39.379174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:39.379248 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:39.379384 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:39.380716 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.381326 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.382660 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.382924 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.383058 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:39.386355 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:39.386358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:39.386361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:39.386388 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:39.388046 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:39.388745 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:39.388751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:39.388753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:39.388848 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:39.389178 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:39.389180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:39.389181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:39.389431 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:39.389651 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:39.390247 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 04:41:39.390371 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260811 04:41:39.391502 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:39.376726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42053 (local address 127.22.144.131:42555)
0811 04:41:39.377320 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:39.377411 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:39.378897 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:39.379598 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:39.379605 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:39.380002 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:39.380528 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:39.380543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.381505 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.381508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:39.382503 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:39.382515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.383103 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.383108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:39.383358 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:39.384058 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:39.384082 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:39.386186 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:39.388188 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:39.388193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:39.388206 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:39.388544 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:39.388933 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:39.388935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:39.388936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:39.389081 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:39.389662 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:39.389668 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:39.389974 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 04:41:39.390156 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 04:41:39.390524 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":129,"threads_started":1}
I20260811 04:41:39.392791 27522 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42053
I20260811 04:41:39.393021 27522 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:39.393714 27522 heartbeater.cc:511] Master 127.22.144.190:42053 requested a full tablet report, sending...
I20260811 04:41:39.394820 27026 ts_manager.cc:194] Registered new tserver with Master: 75065f5104524927b8b91d93ff5f7fd5 (127.22.144.131:37429)
I20260811 04:41:39.395411 27026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:42555
I20260811 04:41:39.399294 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 04:41:39 dist-test-slave-ss4n krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 04:41:39 dist-test-slave-ss4n krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:39.451534 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:39.439328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.22.144.129:34675)
0811 04:41:39.440016 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:39.440023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:39.440048 (+ 25us) server_negotiation.cc:407] Connection header received
0811 04:41:39.440103 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:39.440108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:39.440390 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:39.440529 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:39.441737 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.442794 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.443809 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.444289 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.444472 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:39.447532 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:39.447547 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:39.447573 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:39.447604 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:39.449364 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:39.449917 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:39.449923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:39.449924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:39.450013 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:39.450330 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:39.450334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:39.450340 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:39.450672 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:39.450946 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:39.451141 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 04:41:39.451331 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":279,"threads_started":1}
Aug 11 04:41:39 dist-test-slave-ss4n krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:39.464318 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:39.454795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.22.144.190:42053)
0811 04:41:39.454946 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:39.454950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:39.454966 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:39.455108 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:39.455112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:39.455159 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:39.455232 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:39.455876 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.456413 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.457280 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:39.457572 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:39.457702 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:39.460258 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:39.460261 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:39.460263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:39.460289 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:39.461907 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:39.462985 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:39.462988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:39.462989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:39.463075 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:39.463368 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:39.463370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:39.463371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:39.463614 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:39.463722 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:39.463985 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 04:41:39.464102 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 04:41:39.464900 27026 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56516
I20260811 04:41:39.465730 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27091
I20260811 04:41:39.475381 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27237
I20260811 04:41:39.485253 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27382
I20260811 04:41:39.494359 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 26992
2026-08-11T04:41:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3257 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:40 dist-test-slave-ss4n krb5kdc[27536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:40 dist-test-slave-ss4n krb5kdc[27536](info): set up 2 sockets
Aug 11 04:41:40 dist-test-slave-ss4n krb5kdc[27536](info): commencing operation
krb5kdc: starting...
W20260811 04:41:41.526357 23106 mini_kdc.cc:121] Time spent starting KDC: real 2.005s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:41 dist-test-slave-ss4n krb5kdc[27536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:41.669730 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:37777
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:37777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:41.789433 27550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:41.789757 27550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:41.789803 27550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:41.793834 27550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:41.793895 27550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:41.793917 27550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:41.793933 27550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:41.793949 27550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:41.799372 27550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:37777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:37777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:41.800463 27550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:41.801498 27550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:41.810974 27555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:41.810974 27556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:41.811100 27559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:41.811236 27550 server_base.cc:1034] running on GCE node
I20260811 04:41:41.811867 27550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:41.813159 27550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:41.814390 27550 hybrid_clock.cc:648] HybridClock initialized: now 1786423301814376 us; error 42 us; skew 500 ppm
Aug 11 04:41:41 dist-test-slave-ss4n krb5kdc[27536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423301, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:41.822482 27550 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:41.823357 27550 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:41.823462 27550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:41.823712 27550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:41.826000 27550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "1634310bbecd4c9bb477e8fd79b72305"
format_stamp: "Formatted at 2026-08-11 04:41:41 on dist-test-slave-ss4n"
server_key: "23ae6779463566435adaf3e282ce310c"
server_key_iv: "81ecf2f3b0dc734d115712ad14840faf"
server_key_version: "encryptionkey@0"
I20260811 04:41:41.826610 27550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "1634310bbecd4c9bb477e8fd79b72305"
format_stamp: "Formatted at 2026-08-11 04:41:41 on dist-test-slave-ss4n"
server_key: "23ae6779463566435adaf3e282ce310c"
server_key_iv: "81ecf2f3b0dc734d115712ad14840faf"
server_key_version: "encryptionkey@0"
I20260811 04:41:41.830530 27550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:41.833087 27566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:41.834568 27550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:41:41.834756 27550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "1634310bbecd4c9bb477e8fd79b72305"
format_stamp: "Formatted at 2026-08-11 04:41:41 on dist-test-slave-ss4n"
server_key: "23ae6779463566435adaf3e282ce310c"
server_key_iv: "81ecf2f3b0dc734d115712ad14840faf"
server_key_version: "encryptionkey@0"
I20260811 04:41:41.834901 27550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:41.878139 27550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:41.881983 27550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:41.882277 27550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:41.891160 27550 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:37777
I20260811 04:41:41.891490 27622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:37777 every 8 connection(s)
I20260811 04:41:41.891933 27550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:41.895160 27623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:41.899196 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27550
I20260811 04:41:41.899359 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:41.899657 23106 external_mini_cluster.cc:1428] Setting key 09844d536c1f4c6970f0d9c8a8e41b26
I20260811 04:41:41.902307 27623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: Bootstrap starting.
I20260811 04:41:41.905243 27623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:41.906364 27623 log.cc:824] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:41.909168 27623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: No bootstrap required, opened a new log
Aug 11 04:41:41 dist-test-slave-ss4n krb5kdc[27536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:41.913769 27623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1634310bbecd4c9bb477e8fd79b72305" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 37777 } }
I20260811 04:41:41.914106 27623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:41.914214 27623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1634310bbecd4c9bb477e8fd79b72305, State: Initialized, Role: FOLLOWER
I20260811 04:41:41.914829 27623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1634310bbecd4c9bb477e8fd79b72305" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 37777 } }
I20260811 04:41:41.915062 27623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:41.915170 27623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:41.915298 27623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:41.916754 27623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1634310bbecd4c9bb477e8fd79b72305" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 37777 } }
I20260811 04:41:41.917179 27623 leader_election.cc:302] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1634310bbecd4c9bb477e8fd79b72305; no voters:
I20260811 04:41:41.917791 27623 leader_election.cc:288] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:41.918382 27629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:41.919409 27623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:41.919553 27629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [term 1 LEADER]: Becoming Leader. State: Replica: 1634310bbecd4c9bb477e8fd79b72305, State: Running, Role: LEADER
I20260811 04:41:41.919711 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:41.901248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.22.144.190:37777)
0811 04:41:41.901800 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:41.901828 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:41.901869 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:41:41.902660 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:41.902680 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:41.903070 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:41.903371 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:41.904465 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:41.905824 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:41.907199 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:41.907841 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:41.907992 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:41.911351 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:41.911357 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:41.911372 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:41.911404 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:41.914032 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:41.914764 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:41.914772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:41.914779 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:41.915177 (+ 398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:41.915779 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:41.915784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:41.915786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:41.916630 (+ 844us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:41.916923 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:41.917885 (+ 962us) server_negotiation.cc:299] Negotiation successful
0811 04:41:41.918366 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":281,"threads_started":1}
I20260811 04:41:41.920336 27629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1634310bbecd4c9bb477e8fd79b72305" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 37777 } }
I20260811 04:41:41.923839 27628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1634310bbecd4c9bb477e8fd79b72305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1634310bbecd4c9bb477e8fd79b72305" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 37777 } } }
I20260811 04:41:41.924949 27628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:41.925072 27630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1634310bbecd4c9bb477e8fd79b72305. Latest consensus state: current_term: 1 leader_uuid: "1634310bbecd4c9bb477e8fd79b72305" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1634310bbecd4c9bb477e8fd79b72305" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 37777 } } }
I20260811 04:41:41.925179 27630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305 [sys.catalog]: This master's current role is: LEADER
W20260811 04:41:41.926859 27644 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:41.926932 27644 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:41.927039 27646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:41.927856 27646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:41.934233 27646 catalog_manager.cc:1347] Generated new cluster ID: 09b1238ffbd74bf8813ba3c7da9a1f0b
I20260811 04:41:41.934300 27646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:41.942305 27646 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:41.943336 27646 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:41.950320 27646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: Generated new TSK 0
I20260811 04:41:41.951148 27646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:41.985870 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:37777
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:42.118103 27651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:42.118423 27651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:42.118485 27651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:42.123785 27651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:42.123867 27651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:42.123957 27651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:42.130203 27651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:37777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:42.131505 27651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:42.132666 27651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:42.143445 27657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:42.143543 27656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:42.143455 27660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:42.144335 27651 server_base.cc:1034] running on GCE node
I20260811 04:41:42.144886 27651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:42.145722 27651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:42.146921 27651 hybrid_clock.cc:648] HybridClock initialized: now 1786423302146890 us; error 66 us; skew 500 ppm
Aug 11 04:41:42 dist-test-slave-ss4n krb5kdc[27536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:42.150208 27651 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:42.151126 27651 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:42.151238 27651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:42.151587 27651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:42.153947 27651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "9589ad1dd9d240edaa35df9d645d4dbb"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "23c9fc4e260a21697930bc10f66a1cde"
server_key_iv: "073cb817d922943d55c6981c95e951c2"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.154619 27651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "9589ad1dd9d240edaa35df9d645d4dbb"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "23c9fc4e260a21697930bc10f66a1cde"
server_key_iv: "073cb817d922943d55c6981c95e951c2"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.159446 27651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 04:41:42.162374 27667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.164036 27651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:41:42.164196 27651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "9589ad1dd9d240edaa35df9d645d4dbb"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "23c9fc4e260a21697930bc10f66a1cde"
server_key_iv: "073cb817d922943d55c6981c95e951c2"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.164321 27651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:42.185333 27651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:42.188635 27651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:42.188856 27651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:42.189517 27651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:42.190392 27651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:42.190495 27651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.190577 27651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:42.190634 27651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.200194 27651 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:38879
I20260811 04:41:42.200212 27796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:38879 every 8 connection(s)
I20260811 04:41:42.200830 27651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:42.204774 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27651
I20260811 04:41:42.204932 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:42.205255 23106 external_mini_cluster.cc:1428] Setting key 09e3d6640c200b43531a963adc4036f4
Aug 11 04:41:42 dist-test-slave-ss4n krb5kdc[27536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:42.217773 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.202774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:43631 (local address 127.22.144.190:37777)
0811 04:41:42.202947 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.202952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.204156 (+ 1204us) server_negotiation.cc:407] Connection header received
0811 04:41:42.205428 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.205431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.205531 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.205625 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:42.207169 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.207782 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.209566 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.209973 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.210109 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.213126 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:42.213134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:42.213141 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:42.213178 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:42.215172 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.215793 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.215800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.215804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.215929 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.216345 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.216351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.216355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.216638 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:42.216771 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.217339 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.217529 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 04:41:42.218695 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.203096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:37777 (local address 127.22.144.129:43631)
0811 04:41:42.203974 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.204020 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.205153 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.205866 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.205874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.206339 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:42.206982 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.207004 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.207940 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.207947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.209398 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.209418 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.210180 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.210184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.210347 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.211052 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:42.211081 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:42.212932 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:42.215336 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:42.215342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:42.215356 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:42.215675 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:42.216069 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:42.216072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:42.216073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:42.216231 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:42.216774 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:42.216780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:42.217043 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.217233 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.217606 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":112,"threads_started":1}
I20260811 04:41:42.220234 27797 heartbeater.cc:347] Connected to a master server at 127.22.144.190:37777
I20260811 04:41:42.220464 27797 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:42.221175 27797 heartbeater.cc:511] Master 127.22.144.190:37777 requested a full tablet report, sending...
I20260811 04:41:42.222856 27586 ts_manager.cc:194] Registered new tserver with Master: 9589ad1dd9d240edaa35df9d645d4dbb (127.22.144.129:38879)
I20260811 04:41:42.224005 27586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:43631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:42.241024 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:37777
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:42.360720 27802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:42.361007 27802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:42.361054 27802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:42.365060 27802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:42.365127 27802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:42.365188 27802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:42.370934 27802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:37777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:42.372011 27802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:42.373030 27802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:42.382810 27807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:42.382860 27808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:42.382822 27810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:42.383203 27802 server_base.cc:1034] running on GCE node
I20260811 04:41:42.383695 27802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:42.384438 27802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:42.385735 27802 hybrid_clock.cc:648] HybridClock initialized: now 1786423302385707 us; error 45 us; skew 500 ppm
Aug 11 04:41:42 dist-test-slave-ss4n krb5kdc[27536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:42.388692 27802 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:42.389591 27802 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:42.389698 27802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:42.389923 27802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:42.392042 27802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "b18780314234439997f6ac25be4cbdfa"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "f9aac15711724f33e98a72a7a7740305"
server_key_iv: "c0567a940b07b82427c7bb2b657a0a64"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.392639 27802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "b18780314234439997f6ac25be4cbdfa"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "f9aac15711724f33e98a72a7a7740305"
server_key_iv: "c0567a940b07b82427c7bb2b657a0a64"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.396697 27802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:42.399456 27817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.400805 27802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:42.400947 27802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "b18780314234439997f6ac25be4cbdfa"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "f9aac15711724f33e98a72a7a7740305"
server_key_iv: "c0567a940b07b82427c7bb2b657a0a64"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.401073 27802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:42.419063 27802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:42.423321 27802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:42.423605 27802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:42.424309 27802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:42.425347 27802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:42.425437 27802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.425565 27802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:42.425621 27802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.436200 27802 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:38327
I20260811 04:41:42.436221 27944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:38327 every 8 connection(s)
I20260811 04:41:42.436913 27802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:42.439509 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27802
I20260811 04:41:42.439657 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:42.439985 23106 external_mini_cluster.cc:1428] Setting key d380eb7d3b586519c3a0588d8d5e292f
Aug 11 04:41:42 dist-test-slave-ss4n krb5kdc[27536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:41:42.454069 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.439119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:50589 (local address 127.22.144.190:37777)
0811 04:41:42.439286 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.439293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.440937 (+ 1644us) server_negotiation.cc:407] Connection header received
0811 04:41:42.441984 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.441989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.442049 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.442158 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:42.443508 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.444304 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.446221 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.446448 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.446621 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.449789 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:42.449792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:42.449794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:42.449820 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:42.451442 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.452355 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.452360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.452362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.452448 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.452824 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.452827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.452829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.453035 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:42.453147 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.453757 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.453890 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:41:42.455080 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.439707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:37777 (local address 127.22.144.130:50589)
0811 04:41:42.440800 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.440857 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.441759 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.442331 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.442338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.442782 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:42.443315 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.443330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.444459 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.444465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.446076 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.446092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.446697 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.446704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.446984 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.447682 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:42.447705 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:42.449604 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:42.451734 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:42.451760 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:42.451780 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:42.452241 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:42.452552 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:42.452556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:42.452558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:42.452717 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:42.453148 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:42.453156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:42.453434 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.453701 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.454067 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":165,"threads_started":1}
I20260811 04:41:42.456434 27945 heartbeater.cc:347] Connected to a master server at 127.22.144.190:37777
I20260811 04:41:42.456727 27945 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:42.457228 27945 heartbeater.cc:511] Master 127.22.144.190:37777 requested a full tablet report, sending...
I20260811 04:41:42.458422 27586 ts_manager.cc:194] Registered new tserver with Master: b18780314234439997f6ac25be4cbdfa (127.22.144.130:38327)
I20260811 04:41:42.459036 27586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:50589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:42.474013 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:37777
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:42.590065 27951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:42.590363 27951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:42.590437 27951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:42.594676 27951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:42.594769 27951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:42.594862 27951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:42.600457 27951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:37777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:42.601689 27951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:42.602797 27951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:42.613116 27960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:42.613317 27951 server_base.cc:1034] running on GCE node
W20260811 04:41:42.613168 27957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:42.613121 27958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:42.613903 27951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:42.614552 27951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:42.615739 27951 hybrid_clock.cc:648] HybridClock initialized: now 1786423302615725 us; error 25 us; skew 500 ppm
Aug 11 04:41:42 dist-test-slave-ss4n krb5kdc[27536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:42.619158 27951 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:42.619911 27951 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:42.620021 27951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:42.620311 27951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:42.622776 27951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "ef91e930e4854ffc90ff357fc98535e1"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "aa7264391745227d71135bef1f530a74"
server_key_iv: "6c23e8cfc4ab48d1d50ca5601d6bafce"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.623383 27951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "ef91e930e4854ffc90ff357fc98535e1"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "aa7264391745227d71135bef1f530a74"
server_key_iv: "6c23e8cfc4ab48d1d50ca5601d6bafce"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.627467 27951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:42.630198 27966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.631626 27951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:42.631767 27951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "ef91e930e4854ffc90ff357fc98535e1"
format_stamp: "Formatted at 2026-08-11 04:41:42 on dist-test-slave-ss4n"
server_key: "aa7264391745227d71135bef1f530a74"
server_key_iv: "6c23e8cfc4ab48d1d50ca5601d6bafce"
server_key_version: "encryptionkey@0"
I20260811 04:41:42.631896 27951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:42.663632 27951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:42.667526 27951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:42.667742 27951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:42.668371 27951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:42.669279 27951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:42.669353 27951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.669425 27951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:42.669476 27951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:42.679095 27951 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:34933
I20260811 04:41:42.679112 28093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:34933 every 8 connection(s)
I20260811 04:41:42.679755 27951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:42.682592 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 27951
I20260811 04:41:42.682727 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:42.683043 23106 external_mini_cluster.cc:1428] Setting key 80584e133d6f08575b3971c53579205e
Aug 11 04:41:42 dist-test-slave-ss4n krb5kdc[27536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423302, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:42.694765 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.681601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:34155 (local address 127.22.144.190:37777)
0811 04:41:42.681748 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.681752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.683308 (+ 1556us) server_negotiation.cc:407] Connection header received
0811 04:41:42.684356 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.684359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.684406 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.684631 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:42.686104 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.686741 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.687899 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.688169 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.688277 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.690908 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:42.690912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:42.690914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:42.690939 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:42.692497 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.693063 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.693066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.693067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.693158 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.693625 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.693628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.693629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.693841 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:42.693925 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.694508 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.694629 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 04:41:42.695564 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.682143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:37777 (local address 127.22.144.131:34155)
0811 04:41:42.683124 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.683191 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.684143 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.684725 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.684734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.685278 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:42.685937 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.685953 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.686865 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.686868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.687758 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.687767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.688324 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.688327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.688473 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.689106 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:42.689122 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:42.690741 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:42.692622 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:42.692627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:42.692645 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:42.692921 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:42.693247 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:42.693249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:42.693250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:42.693383 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:42.693937 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:42.693942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:42.694207 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.694365 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.694645 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":146,"threads_started":1}
I20260811 04:41:42.696915 28094 heartbeater.cc:347] Connected to a master server at 127.22.144.190:37777
I20260811 04:41:42.697217 28094 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:42.697791 28094 heartbeater.cc:511] Master 127.22.144.190:37777 requested a full tablet report, sending...
I20260811 04:41:42.698920 27585 ts_manager.cc:194] Registered new tserver with Master: ef91e930e4854ffc90ff357fc98535e1 (127.22.144.131:34933)
I20260811 04:41:42.699539 27585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:34155
I20260811 04:41:42.710024 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:41:42.720706 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.712208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.22.144.190:37777)
0811 04:41:42.712414 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.712419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.712584 (+ 165us) server_negotiation.cc:407] Connection header received
0811 04:41:42.712745 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.712749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.712801 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.712892 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:42.713775 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.714308 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.715199 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.715424 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.715535 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.716933 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:42.716936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:42.716938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:42.716962 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:42.718477 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.718979 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.718989 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.718991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.719070 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:42.719395 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:42.719398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:42.719402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:42.719732 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:42.719884 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.720232 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.720559 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 04:41:42.724085 27585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:42.727195 27585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:42.743377 28108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.736354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38327 (local address 127.0.0.1:53566)
0811 04:41:42.737354 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.737412 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.737584 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.738365 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.738371 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.738395 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:42.738626 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.738635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.740223 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.740226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.741839 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.741849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.743004 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.743010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.743103 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.743125 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.743202 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.743262 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":229,"thread_start_us":149,"threads_started":1}
I20260811 04:41:42.743675 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.736498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.22.144.130:38327)
0811 04:41:42.737914 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.737931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.737969 (+ 38us) server_negotiation.cc:407] Connection header received
0811 04:41:42.738050 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.738054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.738205 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.738374 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:42.738762 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.740109 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.741986 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.742841 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.742965 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.743091 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.743400 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.743523 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":127,"threads_started":1}
I20260811 04:41:42.743872 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.737160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.22.144.131:34933)
0811 04:41:42.737716 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.737726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.738086 (+ 360us) server_negotiation.cc:407] Connection header received
0811 04:41:42.738156 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.738160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.738313 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.738431 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:42.738891 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.740339 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.741965 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.742593 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.742750 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.742901 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.743567 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.743685 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":154,"threads_started":1}
I20260811 04:41:42.744093 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.736326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:38879 (local address 127.0.0.1:37516)
0811 04:41:42.737758 (+ 1432us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.737773 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.737877 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.738533 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.738538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.738571 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:42.738765 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.738773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.740094 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.740098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.741837 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.741849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.743157 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.743160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.743255 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.743279 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.743354 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.743431 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":206,"thread_start_us":75,"threads_started":1}
I20260811 04:41:42.744575 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.737160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.22.144.129:38879)
0811 04:41:42.737574 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.737583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.737877 (+ 294us) server_negotiation.cc:407] Connection header received
0811 04:41:42.737993 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.737998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.738219 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.738370 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:42.738910 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.739964 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.741986 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.742841 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.742965 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.743089 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.744310 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.744414 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":82,"threads_started":1}
I20260811 04:41:42.744234 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.736587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34933 (local address 127.0.0.1:51842)
0811 04:41:42.737876 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.737891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.737986 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.738569 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.738572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.738584 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:42.738770 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.738775 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.740522 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.740667 (+ 145us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.741819 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.741830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.742737 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.742741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.742857 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.742900 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.743025 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.743168 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"mutex_wait_us":162,"thread_start_us":92,"threads_started":1}
I20260811 04:41:42.746917 27870 tablet_service.cc:1467] Processing CreateTablet for tablet bd84cee73b464db597801c6fbf330156 (DEFAULT_TABLE table=test-table [id=9fda437860dd4b2f90ccbdf387f575c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:42.747109 27723 tablet_service.cc:1467] Processing CreateTablet for tablet bd84cee73b464db597801c6fbf330156 (DEFAULT_TABLE table=test-table [id=9fda437860dd4b2f90ccbdf387f575c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:42.748153 27723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd84cee73b464db597801c6fbf330156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:42.748178 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd84cee73b464db597801c6fbf330156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:42.754899 28019 tablet_service.cc:1467] Processing CreateTablet for tablet bd84cee73b464db597801c6fbf330156 (DEFAULT_TABLE table=test-table [id=9fda437860dd4b2f90ccbdf387f575c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:42.756150 28019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd84cee73b464db597801c6fbf330156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:42.756132 28113 tablet_bootstrap.cc:492] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Bootstrap starting.
I20260811 04:41:42.757845 28112 tablet_bootstrap.cc:492] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: Bootstrap starting.
I20260811 04:41:42.758529 28113 tablet_bootstrap.cc:654] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:42.759944 28113 log.cc:824] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:42.760478 28112 tablet_bootstrap.cc:654] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:42.761499 28112 log.cc:824] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:42.762408 28113 tablet_bootstrap.cc:492] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: No bootstrap required, opened a new log
I20260811 04:41:42.762668 28113 ts_tablet_manager.cc:1397] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 04:41:42.763056 28116 tablet_bootstrap.cc:492] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Bootstrap starting.
I20260811 04:41:42.764571 28112 tablet_bootstrap.cc:492] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: No bootstrap required, opened a new log
I20260811 04:41:42.764787 28112 ts_tablet_manager.cc:1397] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 04:41:42.765142 28116 tablet_bootstrap.cc:654] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:42.766714 28116 log.cc:824] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:42.767700 28113 raft_consensus.cc:348] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.768028 28113 raft_consensus.cc:374] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:42.768118 28113 raft_consensus.cc:729] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b18780314234439997f6ac25be4cbdfa, State: Initialized, Role: FOLLOWER
I20260811 04:41:42.768709 28113 consensus_queue.cc:260] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.768932 28116 tablet_bootstrap.cc:492] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: No bootstrap required, opened a new log
I20260811 04:41:42.769122 28116 ts_tablet_manager.cc:1397] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 04:41:42.769948 27945 heartbeater.cc:503] Master 127.22.144.190:37777 was elected leader, sending a full tablet report...
I20260811 04:41:42.770025 28112 raft_consensus.cc:348] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.770327 28112 raft_consensus.cc:374] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:42.770395 28112 raft_consensus.cc:729] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9589ad1dd9d240edaa35df9d645d4dbb, State: Initialized, Role: FOLLOWER
I20260811 04:41:42.770871 28112 consensus_queue.cc:260] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.771135 28113 ts_tablet_manager.cc:1428] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 04:41:42.771759 27797 heartbeater.cc:503] Master 127.22.144.190:37777 was elected leader, sending a full tablet report...
I20260811 04:41:42.771977 28112 ts_tablet_manager.cc:1428] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 04:41:42.773916 28116 raft_consensus.cc:348] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.774326 28116 raft_consensus.cc:374] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:42.774456 28116 raft_consensus.cc:729] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef91e930e4854ffc90ff357fc98535e1, State: Initialized, Role: FOLLOWER
I20260811 04:41:42.775039 28116 consensus_queue.cc:260] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.775775 28094 heartbeater.cc:503] Master 127.22.144.190:37777 was elected leader, sending a full tablet report...
I20260811 04:41:42.776239 28116 ts_tablet_manager.cc:1428] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 04:41:42.878755 28121 raft_consensus.cc:482] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:42.878980 28121 raft_consensus.cc:504] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.880211 28121 leader_election.cc:288] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b18780314234439997f6ac25be4cbdfa (127.22.144.130:38327), 9589ad1dd9d240edaa35df9d645d4dbb (127.22.144.129:38879)
I20260811 04:41:42.885988 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.880497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38327 (local address 127.22.144.131:42093)
0811 04:41:42.880991 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.881002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.881098 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.881368 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.881373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.881386 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:42.881522 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.881531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.882666 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.882670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.884269 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.884281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.885305 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.885308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.885407 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.885435 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.885734 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.885821 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":244,"thread_start_us":144,"threads_started":1}
I20260811 04:41:42.885972 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.880719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:42093 (local address 127.22.144.130:38327)
0811 04:41:42.880865 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.880870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.881096 (+ 226us) server_negotiation.cc:407] Connection header received
0811 04:41:42.881208 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.881211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.881263 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.881342 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:42.881683 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.882533 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.884422 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.885146 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.885304 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.885372 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.885754 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.885828 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:41:42.886142 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.880497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:38879 (local address 127.22.144.131:38985)
0811 04:41:42.880617 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.880635 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:42.880739 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:42.881242 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:42.881244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:42.881260 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:42.881404 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.881411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.882666 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.882670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.884269 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:42.884281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.885468 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.885472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:42.885773 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.885805 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 04:41:42.885890 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 04:41:42.885988 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":223}
I20260811 04:41:42.886196 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.880800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:38985 (local address 127.22.144.129:38879)
0811 04:41:42.880938 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.880942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.880955 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:42.881016 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.881018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.881099 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.881210 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:42.881607 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.882533 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.884422 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.885146 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.885304 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.885372 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.885931 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.886067 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:41:42.886914 27890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd84cee73b464db597801c6fbf330156" candidate_uuid: "ef91e930e4854ffc90ff357fc98535e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b18780314234439997f6ac25be4cbdfa" is_pre_election: true
I20260811 04:41:42.887044 27745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd84cee73b464db597801c6fbf330156" candidate_uuid: "ef91e930e4854ffc90ff357fc98535e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" is_pre_election: true
I20260811 04:41:42.887277 27890 raft_consensus.cc:2454] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef91e930e4854ffc90ff357fc98535e1 in term 0.
I20260811 04:41:42.887409 27745 raft_consensus.cc:2454] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef91e930e4854ffc90ff357fc98535e1 in term 0.
I20260811 04:41:42.888011 27971 leader_election.cc:302] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9589ad1dd9d240edaa35df9d645d4dbb, b18780314234439997f6ac25be4cbdfa, ef91e930e4854ffc90ff357fc98535e1; no voters:
I20260811 04:41:42.888350 28121 raft_consensus.cc:2781] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:42.888460 28121 raft_consensus.cc:482] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:42.888597 28121 raft_consensus.cc:3037] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:42.889801 28121 raft_consensus.cc:504] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.890221 28121 leader_election.cc:288] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [CANDIDATE]: Term 1 election: Requested vote from peers b18780314234439997f6ac25be4cbdfa (127.22.144.130:38327), 9589ad1dd9d240edaa35df9d645d4dbb (127.22.144.129:38879)
I20260811 04:41:42.890620 27745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd84cee73b464db597801c6fbf330156" candidate_uuid: "ef91e930e4854ffc90ff357fc98535e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9589ad1dd9d240edaa35df9d645d4dbb"
I20260811 04:41:42.890641 27890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd84cee73b464db597801c6fbf330156" candidate_uuid: "ef91e930e4854ffc90ff357fc98535e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b18780314234439997f6ac25be4cbdfa"
I20260811 04:41:42.890775 27890 raft_consensus.cc:3037] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:42.890770 27745 raft_consensus.cc:3037] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:42.891935 27890 raft_consensus.cc:2454] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef91e930e4854ffc90ff357fc98535e1 in term 1.
I20260811 04:41:42.892201 27745 raft_consensus.cc:2454] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef91e930e4854ffc90ff357fc98535e1 in term 1.
I20260811 04:41:42.892351 27967 leader_election.cc:302] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b18780314234439997f6ac25be4cbdfa, ef91e930e4854ffc90ff357fc98535e1; no voters:
I20260811 04:41:42.892568 28121 raft_consensus.cc:2781] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:42.892817 28121 raft_consensus.cc:686] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 1 LEADER]: Becoming Leader. State: Replica: ef91e930e4854ffc90ff357fc98535e1, State: Running, Role: LEADER
I20260811 04:41:42.893159 28121 consensus_queue.cc:237] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } }
I20260811 04:41:42.897073 27586 catalog_manager.cc:5495] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef91e930e4854ffc90ff357fc98535e1 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "ef91e930e4854ffc90ff357fc98535e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef91e930e4854ffc90ff357fc98535e1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34933 } health_report { overall_health: HEALTHY } } }
W20260811 04:41:42.930620 28095 tablet.cc:2369] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:42.933409 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.929693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.22.144.131:34933)
0811 04:41:42.929829 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:42.929834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:42.929850 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:42.929983 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:42.929985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:42.930044 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:42.930163 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:42.930449 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.931066 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.931938 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:42.932161 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:42.932281 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:42.932615 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:42.933015 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:42.933153 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:42.933212 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 04:41:42.933266 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 04:41:42.938144 27946 tablet.cc:2369] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:42.939440 27745 raft_consensus.cc:1261] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb [term 1 FOLLOWER]: Refusing update from remote peer ef91e930e4854ffc90ff357fc98535e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:42.939495 27890 raft_consensus.cc:1261] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 1 FOLLOWER]: Refusing update from remote peer ef91e930e4854ffc90ff357fc98535e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:42.940210 28121 consensus_queue.cc:1035] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9589ad1dd9d240edaa35df9d645d4dbb" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 38879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:42.940550 28123 consensus_queue.cc:1035] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b18780314234439997f6ac25be4cbdfa" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:42.946966 28126 mvcc.cc:205] Tried to move back new op lower bound from 7317189848832192512 to 7317189848653692928. Current Snapshot: MvccSnapshot[applied={T|T < 7317189848832192512}]
I20260811 04:41:42.948201 28128 mvcc.cc:205] Tried to move back new op lower bound from 7317189848832192512 to 7317189848653692928. Current Snapshot: MvccSnapshot[applied={T|T < 7317189848832192512}]
I20260811 04:41:42.949576 28127 mvcc.cc:205] Tried to move back new op lower bound from 7317189848832192512 to 7317189848653692928. Current Snapshot: MvccSnapshot[applied={T|T < 7317189848832192512}]
I20260811 04:41:42.957870 27585 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:42.958072 27585 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:42.960500 27585 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1634310bbecd4c9bb477e8fd79b72305: Sending DeleteTablet for 3 replicas of tablet bd84cee73b464db597801c6fbf330156
I20260811 04:41:42.961272 27870 tablet_service.cc:1514] Processing DeleteTablet for tablet bd84cee73b464db597801c6fbf330156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:42 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:53566
I20260811 04:41:42.961390 27723 tablet_service.cc:1514] Processing DeleteTablet for tablet bd84cee73b464db597801c6fbf330156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:42 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:37516
I20260811 04:41:42.961661 28019 tablet_service.cc:1514] Processing DeleteTablet for tablet bd84cee73b464db597801c6fbf330156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:42 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51842
I20260811 04:41:42.961759 28136 tablet_replica.cc:330] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: stopping tablet replica
I20260811 04:41:42.962088 28139 tablet_replica.cc:330] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: stopping tablet replica
I20260811 04:41:42.962086 28136 raft_consensus.cc:2229] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:42.962400 28139 raft_consensus.cc:2229] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:42.962425 28136 raft_consensus.cc:2258] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:42.962774 28139 raft_consensus.cc:2258] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:42.963065 28138 tablet_replica.cc:330] T bd84cee73b464db597801c6fbf330156 P 9589ad1dd9d240edaa35df9d645d4dbb: stopping tablet replica
I20260811 04:41:42.963096 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27651
I20260811 04:41:42.964685 28139 ts_tablet_manager.cc:1905] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:42.967881 28139 ts_tablet_manager.cc:1918] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:42.968016 28139 log.cc:1196] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/bd84cee73b464db597801c6fbf330156
I20260811 04:41:42.968335 28139 ts_tablet_manager.cc:1939] T bd84cee73b464db597801c6fbf330156 P ef91e930e4854ffc90ff357fc98535e1: Deleting consensus metadata
I20260811 04:41:42.969456 27568 catalog_manager.cc:4841] TS ef91e930e4854ffc90ff357fc98535e1 (127.22.144.131:34933): tablet bd84cee73b464db597801c6fbf330156 (table test-table [id=9fda437860dd4b2f90ccbdf387f575c5]) successfully deleted
I20260811 04:41:42.970939 28136 ts_tablet_manager.cc:1905] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 04:41:42.972622 27570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:38879 (error 108)
I20260811 04:41:42.973326 28136 ts_tablet_manager.cc:1918] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:42.973459 28136 log.cc:1196] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/bd84cee73b464db597801c6fbf330156
I20260811 04:41:42.973766 28136 ts_tablet_manager.cc:1939] T bd84cee73b464db597801c6fbf330156 P b18780314234439997f6ac25be4cbdfa: Deleting consensus metadata
I20260811 04:41:42.973759 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.973424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:38879 (local address 127.0.0.1:37526)
0811 04:41:42.973582 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.973650 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:38879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 04:41:42.973964 27570 catalog_manager.cc:4584] TS 9589ad1dd9d240edaa35df9d645d4dbb (127.22.144.129:38879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd84cee73b464db597801c6fbf330156: Network error: Client connection negotiation failed: client connection to 127.22.144.129:38879: connect: Connection refused (error 111)
I20260811 04:41:42.974452 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27802
I20260811 04:41:42.993983 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27951
I20260811 04:41:42.994264 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:42.993939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38327 (local address 127.0.0.1:53568)
0811 04:41:42.994110 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:42.994169 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.130:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 04:41:43.003252 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 27550
2026-08-11T04:41:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3507 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T04:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:43Z Disabled control of system clock
I20260811 04:41:43.038220 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41057
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:39975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 04:41:43.152915 28146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:43.153213 28146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:43.153252 28146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:43.157541 28146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:43.157644 28146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:43.157660 28146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:43.157678 28146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:43.157694 28146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:43.162966 28146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:41057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:43.164290 28146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:43.165760 28146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:43.174595 28155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.174696 28153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.174569 28152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:43.174790 28146 server_base.cc:1034] running on GCE node
I20260811 04:41:43.175503 28146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:43.176697 28146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:43.177866 28146 hybrid_clock.cc:648] HybridClock initialized: now 1786423303177846 us; error 35 us; skew 500 ppm
I20260811 04:41:43.179950 28146 webserver.cc:460] Webserver started at http://127.22.144.190:34523/ using document root <none> and password file <none>
I20260811 04:41:43.180562 28146 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:43.180630 28146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:43.180862 28146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:43.183090 28146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "15a1a7e67d499899357356c62160b5b1"
server_key_iv: "981d9a4a07bc7f0661f4c3f7a0ca091e"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.183739 28146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "15a1a7e67d499899357356c62160b5b1"
server_key_iv: "981d9a4a07bc7f0661f4c3f7a0ca091e"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.187783 28146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 04:41:43.190402 28162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.191759 28146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:41:43.191905 28146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "15a1a7e67d499899357356c62160b5b1"
server_key_iv: "981d9a4a07bc7f0661f4c3f7a0ca091e"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.192022 28146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:43.196712 28146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:43.197340 28146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:43.197590 28146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:43.204788 28146 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:41057
I20260811 04:41:43.204828 28219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:41057 every 8 connection(s)
I20260811 04:41:43.205825 28146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:43.209460 28220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:43.214742 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28146
I20260811 04:41:43.214879 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:43.215268 23106 external_mini_cluster.cc:1428] Setting key 3f8b8dcc5763b2b31f597cec0b4a9f9b
I20260811 04:41:43.216090 28220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d: Bootstrap starting.
I20260811 04:41:43.218822 28220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:43.219806 28220 log.cc:824] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:43.221673 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.216841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.22.144.190:41057)
0811 04:41:43.217645 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.217663 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.217700 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:41:43.218371 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.218388 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.218767 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.219144 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.220045 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.220051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.220065 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.220076 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.220136 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.220153 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.220280 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.220458 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.220755 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":163,"threads_started":1}
I20260811 04:41:43.222236 28220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d: No bootstrap required, opened a new log
I20260811 04:41:43.225844 28220 raft_consensus.cc:348] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41057 } }
I20260811 04:41:43.226079 28220 raft_consensus.cc:374] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:43.226166 28220 raft_consensus.cc:729] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eafa6bcb76da4ee7bca6b5e37df1b72d, State: Initialized, Role: FOLLOWER
I20260811 04:41:43.226634 28220 consensus_queue.cc:260] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41057 } }
I20260811 04:41:43.226781 28220 raft_consensus.cc:388] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:43.226845 28220 raft_consensus.cc:482] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:43.226927 28220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:43.227887 28220 raft_consensus.cc:504] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41057 } }
I20260811 04:41:43.228243 28220 leader_election.cc:302] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eafa6bcb76da4ee7bca6b5e37df1b72d; no voters:
I20260811 04:41:43.228605 28220 leader_election.cc:288] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:43.228785 28225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:43.229050 28225 raft_consensus.cc:686] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [term 1 LEADER]: Becoming Leader. State: Replica: eafa6bcb76da4ee7bca6b5e37df1b72d, State: Running, Role: LEADER
I20260811 04:41:43.229341 28225 consensus_queue.cc:237] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41057 } }
I20260811 04:41:43.230073 28220 sys_catalog.cc:564] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:43.231284 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [sys.catalog]: SysCatalogTable state changed. Reason: New leader eafa6bcb76da4ee7bca6b5e37df1b72d. Latest consensus state: current_term: 1 leader_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41057 } } }
I20260811 04:41:43.231284 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafa6bcb76da4ee7bca6b5e37df1b72d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41057 } } }
I20260811 04:41:43.231433 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:43.231433 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:43.231853 28237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:43.234827 28237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:43.240330 28237 catalog_manager.cc:1347] Generated new cluster ID: a79b3f4eb05449699d34a2c3c84b0455
I20260811 04:41:43.240425 28237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:43.249325 28237 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:43.249774 28237 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:43.255735 28237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d: Generated new TSK 0
I20260811 04:41:43.256469 28237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:41:43.262126 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41057
--builtin_ntp_servers=127.22.144.148:39975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 04:41:43.383790 28247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:43.384057 28247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:43.384097 28247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:43.388111 28247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:43.388176 28247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:43.388232 28247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:43.393205 28247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:43.394357 28247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:43.395918 28247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:43.406148 28253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.406148 28254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.406195 28256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:43.406596 28247 server_base.cc:1034] running on GCE node
I20260811 04:41:43.406986 28247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:43.407575 28247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:43.408790 28247 hybrid_clock.cc:648] HybridClock initialized: now 1786423303408786 us; error 55 us; skew 500 ppm
I20260811 04:41:43.410789 28247 webserver.cc:460] Webserver started at http://127.22.144.129:40441/ using document root <none> and password file <none>
I20260811 04:41:43.411387 28247 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:43.411485 28247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:43.411737 28247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:43.413908 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "4013d8c830cc4d8989fec1f2cb0a2bae"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "6dcdc9930478c21e9861778cb47bca4d"
server_key_iv: "a2fe827760a43d06e9cad1c1f6dea6fd"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.414474 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "4013d8c830cc4d8989fec1f2cb0a2bae"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "6dcdc9930478c21e9861778cb47bca4d"
server_key_iv: "a2fe827760a43d06e9cad1c1f6dea6fd"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.418298 28247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 04:41:43.421036 28263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.422673 28247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:41:43.422796 28247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "4013d8c830cc4d8989fec1f2cb0a2bae"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "6dcdc9930478c21e9861778cb47bca4d"
server_key_iv: "a2fe827760a43d06e9cad1c1f6dea6fd"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.422911 28247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:43.427912 28247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:43.428543 28247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:43.428694 28247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:43.429214 28247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:43.430373 28247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:43.430455 28247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.430518 28247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:43.430580 28247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.440090 28247 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:39077
I20260811 04:41:43.440092 28387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:39077 every 8 connection(s)
I20260811 04:41:43.441174 28247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:43.447634 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.443194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:49685 (local address 127.22.144.190:41057)
0811 04:41:43.443341 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.443345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.444490 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 04:41:43.445870 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.445873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.445922 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.446036 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.447137 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.447141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.447144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.447158 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.447186 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.447194 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.447293 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.447446 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.447495 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:41:43.448521 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.443673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41057 (local address 127.22.144.129:49685)
0811 04:41:43.444357 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.444397 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.445641 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.446219 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.446225 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.446316 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.446869 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.446871 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.446910 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.446977 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.447285 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.447295 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.447302 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.447493 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.447764 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260811 04:41:43.449860 28388 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41057
I20260811 04:41:43.450106 28388 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:43.450410 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28247
I20260811 04:41:43.450536 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:43.450608 28388 heartbeater.cc:511] Master 127.22.144.190:41057 requested a full tablet report, sending...
I20260811 04:41:43.450883 23106 external_mini_cluster.cc:1428] Setting key 47e7e3b92e52e834b24b5da69e51e067
I20260811 04:41:43.452132 28182 ts_manager.cc:194] Registered new tserver with Master: 4013d8c830cc4d8989fec1f2cb0a2bae (127.22.144.129:39077)
I20260811 04:41:43.452888 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41057
--builtin_ntp_servers=127.22.144.148:39975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 04:41:43.570199 28391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:43.570463 28391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:43.570503 28391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:43.574477 28391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:43.574532 28391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:43.574587 28391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:43.579295 28391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:43.580433 28391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:43.581878 28391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:43.592319 28396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.592321 28397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.592347 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:43.592387 28391 server_base.cc:1034] running on GCE node
I20260811 04:41:43.593046 28391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:43.593850 28391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:43.595018 28391 hybrid_clock.cc:648] HybridClock initialized: now 1786423303594984 us; error 48 us; skew 500 ppm
I20260811 04:41:43.597208 28391 webserver.cc:460] Webserver started at http://127.22.144.130:36775/ using document root <none> and password file <none>
I20260811 04:41:43.597867 28391 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:43.597934 28391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:43.598157 28391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:43.600417 28391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "1f5d7874ee71444caf9fa4a77380f535"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "29c3b85718e7ed3e408459a2919c4d3d"
server_key_iv: "b548fdc678c6d244d9a1e3a22219dbb0"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.601019 28391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "1f5d7874ee71444caf9fa4a77380f535"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "29c3b85718e7ed3e408459a2919c4d3d"
server_key_iv: "b548fdc678c6d244d9a1e3a22219dbb0"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.605844 28391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 04:41:43.609293 28405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.611276 28391 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:41:43.611433 28391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "1f5d7874ee71444caf9fa4a77380f535"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "29c3b85718e7ed3e408459a2919c4d3d"
server_key_iv: "b548fdc678c6d244d9a1e3a22219dbb0"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.611567 28391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:43.616633 28391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:43.617276 28391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:43.617467 28391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:43.618070 28391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:43.618993 28391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:43.619071 28391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.619136 28391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:43.619205 28391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.627931 28391 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:39725
I20260811 04:41:43.627946 28527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:39725 every 8 connection(s)
I20260811 04:41:43.628875 28391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:43.630290 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28391
I20260811 04:41:43.630410 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:43.630789 23106 external_mini_cluster.cc:1428] Setting key 03e9927d32cdc7146aae7388bbb66717
I20260811 04:41:43.633044 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41057
--builtin_ntp_servers=127.22.144.148:39975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 04:41:43.636138 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.631000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:54405 (local address 127.22.144.190:41057)
0811 04:41:43.631221 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.631226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.632780 (+ 1554us) server_negotiation.cc:407] Connection header received
0811 04:41:43.634234 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.634241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.634292 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.634392 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.635684 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.635687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.635689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.635700 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.635721 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.635729 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.635805 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.635973 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.636012 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 04:41:43.637027 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.631490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41057 (local address 127.22.144.130:54405)
0811 04:41:43.632638 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.632681 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.634024 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.634603 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.634610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.634726 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.635344 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.635348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.635408 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.635502 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.635818 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.635828 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.635834 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.636025 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.636282 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
I20260811 04:41:43.638298 28529 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41057
I20260811 04:41:43.638532 28529 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:43.638974 28529 heartbeater.cc:511] Master 127.22.144.190:41057 requested a full tablet report, sending...
I20260811 04:41:43.640028 28182 ts_manager.cc:194] Registered new tserver with Master: 1f5d7874ee71444caf9fa4a77380f535 (127.22.144.130:39725)
W20260811 04:41:43.752802 28532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:43.753123 28532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:43.753206 28532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:43.757507 28532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:43.757604 28532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:43.757699 28532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:43.762639 28532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:43.763809 28532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:43.765268 28532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:41:43.775699 28532 server_base.cc:1034] running on GCE node
W20260811 04:41:43.775638 28538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.775650 28537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:43.775638 28540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:43.776283 28532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:43.776911 28532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:43.778115 28532 hybrid_clock.cc:648] HybridClock initialized: now 1786423303778090 us; error 47 us; skew 500 ppm
I20260811 04:41:43.780191 28532 webserver.cc:460] Webserver started at http://127.22.144.131:42487/ using document root <none> and password file <none>
I20260811 04:41:43.780860 28532 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:43.780953 28532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:43.781204 28532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:43.783475 28532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "7b19aeb2ecd14ad28ccc16a4162890fc"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "af72e424f4bfbfcb3c13d9e5057fc735"
server_key_iv: "13932dcf8010ec586215a44b9598a661"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.784056 28532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "7b19aeb2ecd14ad28ccc16a4162890fc"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "af72e424f4bfbfcb3c13d9e5057fc735"
server_key_iv: "13932dcf8010ec586215a44b9598a661"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.787968 28532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:43.790663 28546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.791929 28532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:41:43.792075 28532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "7b19aeb2ecd14ad28ccc16a4162890fc"
format_stamp: "Formatted at 2026-08-11 04:41:43 on dist-test-slave-ss4n"
server_key: "af72e424f4bfbfcb3c13d9e5057fc735"
server_key_iv: "13932dcf8010ec586215a44b9598a661"
server_key_version: "encryptionkey@0"
I20260811 04:41:43.792197 28532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:43.797542 28532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:43.798241 28532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:43.798434 28532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:43.798985 28532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:43.799925 28532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:43.799996 28532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.800066 28532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:43.800112 28532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:43.810428 28532 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:43191
I20260811 04:41:43.810451 28671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:43191 every 8 connection(s)
I20260811 04:41:43.811594 28532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:43.818410 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.813668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:43125 (local address 127.22.144.190:41057)
0811 04:41:43.813812 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.813815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.815024 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 04:41:43.816529 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.816533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.816580 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.816670 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.817903 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.817907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.817910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.817923 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.817951 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.817961 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.818059 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.818226 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.818274 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 04:41:43.819242 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.814058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41057 (local address 127.22.144.131:43125)
0811 04:41:43.814808 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.814869 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.816271 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.816851 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.816857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.816955 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.817623 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.817624 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.817666 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.817731 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.818052 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.818062 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.818071 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.818258 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.818494 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260811 04:41:43.820374 28672 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41057
I20260811 04:41:43.820607 28672 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:43.820986 28672 heartbeater.cc:511] Master 127.22.144.190:41057 requested a full tablet report, sending...
I20260811 04:41:43.821305 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28532
I20260811 04:41:43.821532 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:43.821898 23106 external_mini_cluster.cc:1428] Setting key 8558ce0ede9595e11639f3cf2f55ed1f
I20260811 04:41:43.822113 28182 ts_manager.cc:194] Registered new tserver with Master: 7b19aeb2ecd14ad28ccc16a4162890fc (127.22.144.131:43191)
I20260811 04:41:43.824127 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:41:43.828150 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.826197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.22.144.190:41057)
0811 04:41:43.826342 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.826346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.826573 (+ 227us) server_negotiation.cc:407] Connection header received
0811 04:41:43.826667 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.826670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.826705 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.826814 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.827610 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.827613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.827615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.827627 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.827648 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.827657 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.827748 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.827873 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.827955 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:41:43.831813 28182 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:43.835233 28182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:43.848927 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.846096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:39725 (local address 127.0.0.1:45138)
0811 04:41:43.846516 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.846530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.846604 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.847152 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.847159 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.847244 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.848061 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.848063 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.848074 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.848105 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.848544 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.848554 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.848561 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.848679 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.848776 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":116,"threads_started":1}
I20260811 04:41:43.849133 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.846298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.22.144.130:39725)
0811 04:41:43.846710 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.846726 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.846747 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:41:43.846798 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.846803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.846957 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.847108 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.848312 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.848316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.848325 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.848334 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.848370 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.848395 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.848498 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.848727 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.848808 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":101,"threads_started":1}
I20260811 04:41:43.849257 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.845013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:43191 (local address 127.0.0.1:34494)
0811 04:41:43.846100 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.846114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.846209 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.847419 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.847425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.847441 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.848388 (+ 947us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.848390 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.848399 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.848427 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.848974 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.848978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.848981 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.849059 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.849101 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":127,"threads_started":1}
I20260811 04:41:43.849567 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.844908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39077 (local address 127.0.0.1:59052)
0811 04:41:43.845729 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.845790 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.845904 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.847301 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.847304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.847322 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.847996 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.848001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.848027 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.848110 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.848544 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.848561 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.848564 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.848679 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.849395 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":628,"thread_start_us":97,"threads_started":1}
I20260811 04:41:43.849733 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.845176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34494 (local address 127.22.144.131:43191)
0811 04:41:43.845918 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.845952 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.846328 (+ 376us) server_negotiation.cc:407] Connection header received
0811 04:41:43.846399 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.846406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.846606 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.846749 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.848712 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.848718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.848729 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.848743 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.848790 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.848829 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.848930 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.849437 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.849553 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":110,"threads_started":1}
I20260811 04:41:43.849048 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.845655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.22.144.129:39077)
0811 04:41:43.846110 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.846123 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.846149 (+ 26us) server_negotiation.cc:407] Connection header received
0811 04:41:43.846208 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.846215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.846427 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.846586 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.848280 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.848287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.848300 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.848313 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.848363 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.848395 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.848498 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.848813 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.848883 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":125,"threads_started":1}
I20260811 04:41:43.852327 28457 tablet_service.cc:1467] Processing CreateTablet for tablet ecdff07f90534c328f5abc668b8c69ff (DEFAULT_TABLE table=test-table [id=4d487cccddba4e0e8b6f160dd1714e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:43.852475 28599 tablet_service.cc:1467] Processing CreateTablet for tablet ecdff07f90534c328f5abc668b8c69ff (DEFAULT_TABLE table=test-table [id=4d487cccddba4e0e8b6f160dd1714e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:43.852438 28315 tablet_service.cc:1467] Processing CreateTablet for tablet ecdff07f90534c328f5abc668b8c69ff (DEFAULT_TABLE table=test-table [id=4d487cccddba4e0e8b6f160dd1714e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:43.853363 28457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecdff07f90534c328f5abc668b8c69ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:43.853514 28315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecdff07f90534c328f5abc668b8c69ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:43.853514 28599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecdff07f90534c328f5abc668b8c69ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:43.859774 28690 tablet_bootstrap.cc:492] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Bootstrap starting.
I20260811 04:41:43.861308 28692 tablet_bootstrap.cc:492] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae: Bootstrap starting.
I20260811 04:41:43.861809 28691 tablet_bootstrap.cc:492] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Bootstrap starting.
I20260811 04:41:43.862442 28690 tablet_bootstrap.cc:654] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:43.863368 28692 tablet_bootstrap.cc:654] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:43.863358 28690 log.cc:824] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:43.863885 28691 tablet_bootstrap.cc:654] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:43.864193 28692 log.cc:824] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:43.864701 28691 log.cc:824] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:43.865545 28690 tablet_bootstrap.cc:492] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: No bootstrap required, opened a new log
I20260811 04:41:43.865751 28690 ts_tablet_manager.cc:1397] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 04:41:43.866101 28692 tablet_bootstrap.cc:492] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae: No bootstrap required, opened a new log
I20260811 04:41:43.866300 28692 ts_tablet_manager.cc:1397] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 04:41:43.866660 28691 tablet_bootstrap.cc:492] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: No bootstrap required, opened a new log
I20260811 04:41:43.866834 28691 ts_tablet_manager.cc:1397] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 04:41:43.870335 28692 raft_consensus.cc:348] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.870620 28692 raft_consensus.cc:374] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:43.870678 28692 raft_consensus.cc:729] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4013d8c830cc4d8989fec1f2cb0a2bae, State: Initialized, Role: FOLLOWER
I20260811 04:41:43.870573 28690 raft_consensus.cc:348] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.870849 28690 raft_consensus.cc:374] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:43.870994 28690 raft_consensus.cc:729] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5d7874ee71444caf9fa4a77380f535, State: Initialized, Role: FOLLOWER
I20260811 04:41:43.871205 28692 consensus_queue.cc:260] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.871378 28691 raft_consensus.cc:348] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.871547 28690 consensus_queue.cc:260] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.871680 28691 raft_consensus.cc:374] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:43.871740 28691 raft_consensus.cc:729] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b19aeb2ecd14ad28ccc16a4162890fc, State: Initialized, Role: FOLLOWER
I20260811 04:41:43.872149 28388 heartbeater.cc:503] Master 127.22.144.190:41057 was elected leader, sending a full tablet report...
I20260811 04:41:43.872208 28691 consensus_queue.cc:260] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.872393 28692 ts_tablet_manager.cc:1428] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:41:43.872555 28690 ts_tablet_manager.cc:1428] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260811 04:41:43.872869 28529 heartbeater.cc:503] Master 127.22.144.190:41057 was elected leader, sending a full tablet report...
I20260811 04:41:43.872962 28672 heartbeater.cc:503] Master 127.22.144.190:41057 was elected leader, sending a full tablet report...
I20260811 04:41:43.873124 28691 ts_tablet_manager.cc:1428] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260811 04:41:43.874861 28699 raft_consensus.cc:482] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:43.875031 28699 raft_consensus.cc:504] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.876775 28699 leader_election.cc:288] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4013d8c830cc4d8989fec1f2cb0a2bae (127.22.144.129:39077), 7b19aeb2ecd14ad28ccc16a4162890fc (127.22.144.131:43191)
I20260811 04:41:43.878352 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.876435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39077 (local address 127.22.144.130:37165)
0811 04:41:43.876653 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.876670 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.876742 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.877029 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.877032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.877048 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.877769 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.877771 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.877782 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.877812 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.878106 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.878109 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.878112 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.878195 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.878232 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260811 04:41:43.879309 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.876621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:37165 (local address 127.22.144.129:39077)
0811 04:41:43.876734 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.876738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.876754 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:43.876869 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.876873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.876913 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.877011 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.877941 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.877944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.877947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.877959 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.877979 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.877988 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.878063 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.879123 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.879167 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 04:41:43.879833 28530 tablet.cc:2369] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:43.879865 28336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecdff07f90534c328f5abc668b8c69ff" candidate_uuid: "1f5d7874ee71444caf9fa4a77380f535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" is_pre_election: true
I20260811 04:41:43.880157 28336 raft_consensus.cc:2454] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5d7874ee71444caf9fa4a77380f535 in term 0.
I20260811 04:41:43.880618 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.877551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:36971 (local address 127.22.144.131:43191)
0811 04:41:43.877681 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.877688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.878675 (+ 987us) server_negotiation.cc:407] Connection header received
0811 04:41:43.878717 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.878720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.878753 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.878823 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.880153 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.880156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.880159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.880171 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.880191 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.880200 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.880308 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.880448 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.880492 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 04:41:43.880594 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.877401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:43191 (local address 127.22.144.130:36971)
0811 04:41:43.878501 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.878514 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.878595 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.879478 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:43.879481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:43.879495 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:43.879989 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:41:43.879993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:43.880010 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:41:43.880042 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:43.880349 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:43.880352 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:43.880355 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 04:41:43.880440 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 04:41:43.880475 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":288,"threads_started":1}
I20260811 04:41:43.880591 28409 leader_election.cc:302] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7874ee71444caf9fa4a77380f535, 4013d8c830cc4d8989fec1f2cb0a2bae; no voters:
I20260811 04:41:43.880862 28699 raft_consensus.cc:2781] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:43.880959 28699 raft_consensus.cc:482] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:43.881019 28699 raft_consensus.cc:3037] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:43.881323 28620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecdff07f90534c328f5abc668b8c69ff" candidate_uuid: "1f5d7874ee71444caf9fa4a77380f535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" is_pre_election: true
I20260811 04:41:43.881625 28620 raft_consensus.cc:2454] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5d7874ee71444caf9fa4a77380f535 in term 0.
I20260811 04:41:43.882392 28699 raft_consensus.cc:504] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.882771 28699 leader_election.cc:288] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [CANDIDATE]: Term 1 election: Requested vote from peers 4013d8c830cc4d8989fec1f2cb0a2bae (127.22.144.129:39077), 7b19aeb2ecd14ad28ccc16a4162890fc (127.22.144.131:43191)
I20260811 04:41:43.883098 28336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecdff07f90534c328f5abc668b8c69ff" candidate_uuid: "1f5d7874ee71444caf9fa4a77380f535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae"
I20260811 04:41:43.883152 28620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecdff07f90534c328f5abc668b8c69ff" candidate_uuid: "1f5d7874ee71444caf9fa4a77380f535" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc"
I20260811 04:41:43.883219 28336 raft_consensus.cc:3037] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:43.883299 28620 raft_consensus.cc:3037] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:43.884434 28336 raft_consensus.cc:2454] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5d7874ee71444caf9fa4a77380f535 in term 1.
I20260811 04:41:43.884586 28620 raft_consensus.cc:2454] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5d7874ee71444caf9fa4a77380f535 in term 1.
I20260811 04:41:43.884892 28409 leader_election.cc:302] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7874ee71444caf9fa4a77380f535, 4013d8c830cc4d8989fec1f2cb0a2bae; no voters:
I20260811 04:41:43.885111 28699 raft_consensus.cc:2781] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:43.885341 28699 raft_consensus.cc:686] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 1 LEADER]: Becoming Leader. State: Replica: 1f5d7874ee71444caf9fa4a77380f535, State: Running, Role: LEADER
I20260811 04:41:43.885675 28699 consensus_queue.cc:237] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } }
I20260811 04:41:43.889015 28181 catalog_manager.cc:5495] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f5d7874ee71444caf9fa4a77380f535 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "1f5d7874ee71444caf9fa4a77380f535" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5d7874ee71444caf9fa4a77380f535" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 39725 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:43.901603 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:43.899704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.22.144.130:39725)
0811 04:41:43.899849 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:43.899853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:43.899867 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:43.900022 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:43.900025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:43.900100 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:43.900172 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:43.901114 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:43.901117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:43.901119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:43.901129 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:43.901145 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:43.901153 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:43.901228 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:43.901360 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 04:41:43.901394 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:41:43.907701 28336 raft_consensus.cc:1261] T ecdff07f90534c328f5abc668b8c69ff P 4013d8c830cc4d8989fec1f2cb0a2bae [term 1 FOLLOWER]: Refusing update from remote peer 1f5d7874ee71444caf9fa4a77380f535: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:43.907701 28620 raft_consensus.cc:1261] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 1 FOLLOWER]: Refusing update from remote peer 1f5d7874ee71444caf9fa4a77380f535: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:43.908443 28699 consensus_queue.cc:1035] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b19aeb2ecd14ad28ccc16a4162890fc" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:43.908658 28702 consensus_queue.cc:1035] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4013d8c830cc4d8989fec1f2cb0a2bae" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:43.915838 28705 mvcc.cc:205] Tried to move back new op lower bound from 7317189852798189568 to 7317189852718329856. Current Snapshot: MvccSnapshot[applied={T|T < 7317189852798189568}]
I20260811 04:41:43.917524 28706 mvcc.cc:205] Tried to move back new op lower bound from 7317189852798189568 to 7317189852718329856. Current Snapshot: MvccSnapshot[applied={T|T < 7317189852798189568}]
I20260811 04:41:43.922540 28707 mvcc.cc:205] Tried to move back new op lower bound from 7317189852798189568 to 7317189852718329856. Current Snapshot: MvccSnapshot[applied={T|T < 7317189852798189568}]
I20260811 04:41:43.928282 28181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:43.928479 28181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:43.931342 28181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eafa6bcb76da4ee7bca6b5e37df1b72d: Sending DeleteTablet for 3 replicas of tablet ecdff07f90534c328f5abc668b8c69ff
I20260811 04:41:43.932397 28315 tablet_service.cc:1514] Processing DeleteTablet for tablet ecdff07f90534c328f5abc668b8c69ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:43 UTC) from {username='slave'} at 127.0.0.1:59052
I20260811 04:41:43.932590 28599 tablet_service.cc:1514] Processing DeleteTablet for tablet ecdff07f90534c328f5abc668b8c69ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:43 UTC) from {username='slave'} at 127.0.0.1:34494
I20260811 04:41:43.932706 28457 tablet_service.cc:1514] Processing DeleteTablet for tablet ecdff07f90534c328f5abc668b8c69ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:43 UTC) from {username='slave'} at 127.0.0.1:45138
I20260811 04:41:43.933286 28717 tablet_replica.cc:330] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: stopping tablet replica
I20260811 04:41:43.933703 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28247
I20260811 04:41:43.933703 28716 tablet_replica.cc:330] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: stopping tablet replica
I20260811 04:41:43.934573 28716 raft_consensus.cc:2229] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:43.935338 28716 raft_consensus.cc:2258] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:43.941938 28717 raft_consensus.cc:2229] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:43.942428 28717 raft_consensus.cc:2258] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:43.942456 28716 ts_tablet_manager.cc:1905] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:43.943934 28717 ts_tablet_manager.cc:1905] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:43.946388 28716 ts_tablet_manager.cc:1918] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:43.946630 28716 log.cc:1196] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/ecdff07f90534c328f5abc668b8c69ff
I20260811 04:41:43.946744 28717 ts_tablet_manager.cc:1918] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:43.946862 28717 log.cc:1196] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/ecdff07f90534c328f5abc668b8c69ff
I20260811 04:41:43.947206 28717 ts_tablet_manager.cc:1939] T ecdff07f90534c328f5abc668b8c69ff P 1f5d7874ee71444caf9fa4a77380f535: Deleting consensus metadata
I20260811 04:41:43.947214 28716 ts_tablet_manager.cc:1939] T ecdff07f90534c328f5abc668b8c69ff P 7b19aeb2ecd14ad28ccc16a4162890fc: Deleting consensus metadata
W20260811 04:41:43.948457 28165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:39077 (error 108)
I20260811 04:41:43.949083 28166 catalog_manager.cc:4841] TS 7b19aeb2ecd14ad28ccc16a4162890fc (127.22.144.131:43191): tablet ecdff07f90534c328f5abc668b8c69ff (table test-table [id=4d487cccddba4e0e8b6f160dd1714e2f]) successfully deleted
I20260811 04:41:43.949375 28166 catalog_manager.cc:4841] TS 1f5d7874ee71444caf9fa4a77380f535 (127.22.144.130:39725): tablet ecdff07f90534c328f5abc668b8c69ff (table test-table [id=4d487cccddba4e0e8b6f160dd1714e2f]) successfully deleted
W20260811 04:41:43.949986 28683 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:43.949344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39077 (local address 127.0.0.1:59064)
0811 04:41:43.949496 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:43.949515 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:43.949606 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:43.949860 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:39077: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 04:41:43.950100 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28391
W20260811 04:41:43.950174 28165 catalog_manager.cc:4584] TS 4013d8c830cc4d8989fec1f2cb0a2bae (127.22.144.129:39077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ecdff07f90534c328f5abc668b8c69ff: Network error: Client connection negotiation failed: client connection to 127.22.144.129:39077: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 04:41:43.960907 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28532
I20260811 04:41:43.969831 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28146
2026-08-11T04:41:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (965 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 04:41:43.995211 23106 mini_oidc.cc:152] Starting JWKS server
I20260811 04:41:43.996992 23106 webserver.cc:460] Webserver started at https://127.0.0.1:40541/ using document root <none> and password file <none>
I20260811 04:41:43.997200 23106 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 04:41:43.997532 23106 webserver.cc:460] Webserver started at http://127.0.0.1:40995/ using document root <none> and password file <none>
2026-08-11T04:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:44Z Disabled control of system clock
I20260811 04:41:44.008513 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35419
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:35419
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40541/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/testchainca.cert with env {}
W20260811 04:41:44.125398 28725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:44.125765 28725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:44.125808 28725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:44.129849 28725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:44.129907 28725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:44.129923 28725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:44.129940 28725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:44.129956 28725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 04:41:44.130023 28725 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 04:41:44.130046 28725 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40541/jwks.json
I20260811 04:41:44.135514 28725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:35419
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40541/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:44.136641 28725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:44.137794 28725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:44.147535 28731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:44.147722 28730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:44.147535 28733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:44.147910 28725 server_base.cc:1034] running on GCE node
I20260811 04:41:44.148761 28725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:44.150321 28725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:44.151540 28725 hybrid_clock.cc:648] HybridClock initialized: now 1786423304151500 us; error 50 us; skew 500 ppm
I20260811 04:41:44.153935 28725 webserver.cc:460] Webserver started at http://127.22.144.190:42977/ using document root <none> and password file <none>
I20260811 04:41:44.154524 28725 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:44.154621 28725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:44.154801 28725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:44.157002 28725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "a35ad06371e34c8a939de9b82d78647e"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "f919a485b811d290ead83f863deb2b7d"
server_key_iv: "b0d54f4f42d628f07bcc3d36afe6878f"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.157630 28725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "a35ad06371e34c8a939de9b82d78647e"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "f919a485b811d290ead83f863deb2b7d"
server_key_iv: "b0d54f4f42d628f07bcc3d36afe6878f"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.161779 28725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:41:44.164637 28743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:44.166121 28725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:44.166234 28725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "a35ad06371e34c8a939de9b82d78647e"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "f919a485b811d290ead83f863deb2b7d"
server_key_iv: "b0d54f4f42d628f07bcc3d36afe6878f"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.166342 28725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:44.179673 28725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:44.180395 28725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:44.180565 28725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:44.188683 28725 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:35419
I20260811 04:41:44.188706 28798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:35419 every 8 connection(s)
I20260811 04:41:44.189850 28725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:44.193220 28800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:44.195569 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28725
I20260811 04:41:44.195696 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:44.195986 23106 external_mini_cluster.cc:1428] Setting key d3338eaf923bf8bac0f215ac17c10157
I20260811 04:41:44.199630 28800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e: Bootstrap starting.
I20260811 04:41:44.201946 28800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:44.202718 28800 log.cc:824] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:44.205127 28800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e: No bootstrap required, opened a new log
I20260811 04:41:44.206976 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:44.197619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.22.144.190:35419)
0811 04:41:44.198501 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:44.198522 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:44.198560 (+ 38us) server_negotiation.cc:407] Connection header received
0811 04:41:44.199256 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:44.199275 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:44.199849 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:44.200232 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:44.200719 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.201906 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.202927 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.203492 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.203695 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.204564 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:44.204570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:44.204585 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:44.205100 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:44.205176 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:44.205205 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:44.205348 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:44.205603 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 04:41:44.205932 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":163,"threads_started":1}
I20260811 04:41:44.209965 28800 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35ad06371e34c8a939de9b82d78647e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35419 } }
I20260811 04:41:44.210170 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:41:44.210238 28800 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:44.210321 28800 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35ad06371e34c8a939de9b82d78647e, State: Initialized, Role: FOLLOWER
I20260811 04:41:44.210727 28800 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35ad06371e34c8a939de9b82d78647e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35419 } }
I20260811 04:41:44.210845 28800 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:44.210932 28800 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:44.211021 28800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:44.212141 28800 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35ad06371e34c8a939de9b82d78647e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35419 } }
I20260811 04:41:44.212487 28800 leader_election.cc:302] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35ad06371e34c8a939de9b82d78647e; no voters:
I20260811 04:41:44.212807 28800 leader_election.cc:288] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:44.212929 28805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:44.213191 28805 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [term 1 LEADER]: Becoming Leader. State: Replica: a35ad06371e34c8a939de9b82d78647e, State: Running, Role: LEADER
I20260811 04:41:44.213773 28805 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35ad06371e34c8a939de9b82d78647e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35419 } }
I20260811 04:41:44.214217 28800 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:44.215413 28807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35ad06371e34c8a939de9b82d78647e. Latest consensus state: current_term: 1 leader_uuid: "a35ad06371e34c8a939de9b82d78647e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35ad06371e34c8a939de9b82d78647e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35419 } } }
I20260811 04:41:44.215551 28807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:44.215963 28815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:44.215941 28806 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35ad06371e34c8a939de9b82d78647e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35ad06371e34c8a939de9b82d78647e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35419 } } }
I20260811 04:41:44.216141 28806 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:44.218937 28815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:44.224316 28815 catalog_manager.cc:1347] Generated new cluster ID: 8d78e5d7ef4843b28c7cf86130b0c853
I20260811 04:41:44.224391 28815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:44.234122 28815 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:44.235044 28815 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:44.244364 28815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35ad06371e34c8a939de9b82d78647e: Generated new TSK 0
I20260811 04:41:44.245271 28815 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 04:41:44.286283 28833 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 04:41:44.286460 28833 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 04:41:44.287034 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:44.261539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.22.144.190:35419)
0811 04:41:44.261696 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:44.261701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:44.261919 (+ 218us) server_negotiation.cc:407] Connection header received
0811 04:41:44.262122 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:44.262124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:44.262188 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:44.262274 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0811 04:41:44.262640 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.263252 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.264519 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.264789 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.264883 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.265287 (+ 404us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 04:41:44.286444 (+ 21157us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40541/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 04:41:44.286882 (+ 438us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40541/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 04:41:44.287137 28803 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40541/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 04:41:44.288368 28830 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:44.261411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:35419 (local address 127.0.0.1:47574)
0811 04:41:44.261825 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:44.261841 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:44.261978 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:44.262325 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:44.262329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:44.262338 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 04:41:44.262472 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:44.262479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.263385 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.263391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:44.264373 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:44.264384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.264915 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.264919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:44.265023 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.265079 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 04:41:44.286802 (+ 21723us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40541/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 04:41:44.287074 (+ 272us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.144.190:35419: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40541/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":144,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.144.190:35419: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40541/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 04:41:44.289197 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28725
2026-08-11T04:41:44Z chronyd exiting
I20260811 04:41:44.330446 23106 test_util.cc:175] -----------------------------------------------
I20260811 04:41:44.330646 23106 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786423274535345-23106-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (337 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 04:41:44.331982 23106 mini_oidc.cc:152] Starting JWKS server
I20260811 04:41:44.332994 23106 webserver.cc:460] Webserver started at https://127.0.0.1:35993/ using document root <none> and password file <none>
I20260811 04:41:44.333156 23106 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 04:41:44.333537 23106 webserver.cc:460] Webserver started at http://127.0.0.1:35145/ using document root <none> and password file <none>
2026-08-11T04:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:44Z Disabled control of system clock
I20260811 04:41:44.344743 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40221
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:39671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:40221
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35993/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/testchainca.cert with env {}
W20260811 04:41:44.458606 28840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:44.458866 28840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:44.458906 28840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:44.462956 28840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:44.463016 28840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:44.463032 28840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:44.463048 28840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:44.463065 28840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 04:41:44.463081 28840 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 04:41:44.463135 28840 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35993/jwks.json
I20260811 04:41:44.468551 28840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:40221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35993/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:44.469784 28840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:44.470845 28840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:44.479987 28846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:44.479981 28848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:44.480000 28845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:44.480417 28840 server_base.cc:1034] running on GCE node
I20260811 04:41:44.481029 28840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:44.482352 28840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:44.483529 28840 hybrid_clock.cc:648] HybridClock initialized: now 1786423304483511 us; error 27 us; skew 500 ppm
I20260811 04:41:44.485627 28840 webserver.cc:460] Webserver started at http://127.22.144.190:37071/ using document root <none> and password file <none>
I20260811 04:41:44.486239 28840 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:44.486307 28840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:44.486613 28840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:44.488902 28840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "7db04418ab1143b28027e98a0e435b3d"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "f9f9627a58f963c333f32dd182d2b63b"
server_key_iv: "f7073903c0686547d3520be8391277fb"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.489575 28840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "7db04418ab1143b28027e98a0e435b3d"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "f9f9627a58f963c333f32dd182d2b63b"
server_key_iv: "f7073903c0686547d3520be8391277fb"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.493726 28840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:41:44.496440 28854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:44.497937 28840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:44.498078 28840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "7db04418ab1143b28027e98a0e435b3d"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "f9f9627a58f963c333f32dd182d2b63b"
server_key_iv: "f7073903c0686547d3520be8391277fb"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.498205 28840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:44.517149 28840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:44.518013 28840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:44.518242 28840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:44.525971 28840 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:40221
I20260811 04:41:44.525991 28914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:40221 every 8 connection(s)
I20260811 04:41:44.526973 28840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:44.530438 28915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:44.531816 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28840
I20260811 04:41:44.531955 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:44.532358 23106 external_mini_cluster.cc:1428] Setting key d3d3485072d349e919d907fba8f89c11
I20260811 04:41:44.537365 28915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d: Bootstrap starting.
I20260811 04:41:44.539780 28915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:44.540728 28915 log.cc:824] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:44.542682 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:44.534018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.22.144.190:40221)
0811 04:41:44.534746 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:44.534763 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:44.534799 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:41:44.535458 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:44.535475 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:44.535864 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:44.536213 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:44.536668 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.537735 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.538648 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.539131 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.539312 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.540308 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:44.540315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:44.540331 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:44.540859 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:44.540950 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:44.540981 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:44.541103 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:44.541336 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 04:41:44.541680 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":151,"threads_started":1}
I20260811 04:41:44.543288 28915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d: No bootstrap required, opened a new log
I20260811 04:41:44.545790 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:41:44.547775 28915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db04418ab1143b28027e98a0e435b3d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40221 } }
I20260811 04:41:44.548017 28915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:44.548102 28915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db04418ab1143b28027e98a0e435b3d, State: Initialized, Role: FOLLOWER
I20260811 04:41:44.548564 28915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db04418ab1143b28027e98a0e435b3d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40221 } }
I20260811 04:41:44.548722 28915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:44.548801 28915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:44.548946 28915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:44.550134 28915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db04418ab1143b28027e98a0e435b3d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40221 } }
I20260811 04:41:44.550513 28915 leader_election.cc:302] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7db04418ab1143b28027e98a0e435b3d; no voters:
I20260811 04:41:44.550858 28915 leader_election.cc:288] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:44.551065 28921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:44.551323 28921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [term 1 LEADER]: Becoming Leader. State: Replica: 7db04418ab1143b28027e98a0e435b3d, State: Running, Role: LEADER
I20260811 04:41:44.551644 28921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db04418ab1143b28027e98a0e435b3d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40221 } }
I20260811 04:41:44.552188 28915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:44.553359 28923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7db04418ab1143b28027e98a0e435b3d. Latest consensus state: current_term: 1 leader_uuid: "7db04418ab1143b28027e98a0e435b3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db04418ab1143b28027e98a0e435b3d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40221 } } }
I20260811 04:41:44.553540 28923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:44.553861 28922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7db04418ab1143b28027e98a0e435b3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7db04418ab1143b28027e98a0e435b3d" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40221 } } }
I20260811 04:41:44.553963 28930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:44.553984 28922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:44.557657 28930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:44.563685 28930 catalog_manager.cc:1347] Generated new cluster ID: f16ab566bd0d4367baac67a148e36266
I20260811 04:41:44.563772 28930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:44.572537 28930 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:44.573434 28930 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:44.584126 28930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7db04418ab1143b28027e98a0e435b3d: Generated new TSK 0
I20260811 04:41:44.584797 28930 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 04:41:44.608180 28946 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 04:41:44.608283 28946 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260811 04:41:44.609177 28945 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:44.593068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40221 (local address 127.0.0.1:57956)
0811 04:41:44.593369 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:44.593381 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:44.593528 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:44.593892 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:44.593895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:44.593904 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 04:41:44.594015 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:44.594022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.594797 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.594801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:44.595441 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:44.595449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.595932 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.595936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:44.596039 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.596065 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 04:41:44.608821 (+ 12756us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 04:41:44.608997 (+ 176us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.144.190:40221: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":114,"threads_started":1}
I20260811 04:41:44.609346 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:44.593147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.22.144.190:40221)
0811 04:41:44.593268 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:44.593275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:44.593458 (+ 183us) server_negotiation.cc:407] Connection header received
0811 04:41:44.593725 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:44.593727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:44.593775 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:44.593892 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0811 04:41:44.594138 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.594679 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.595576 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.595801 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.595909 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.596244 (+ 335us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 04:41:44.608606 (+ 12362us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 04:41:44.608909 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 04:41:44.609412 28919 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35993/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 04:41:44.609699 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28840
2026-08-11T04:41:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (321 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 04:41:44.653038 23106 mini_oidc.cc:152] Starting JWKS server
I20260811 04:41:44.653975 23106 webserver.cc:460] Webserver started at https://127.0.0.1:38441/ using document root <none> and password file <none>
I20260811 04:41:44.654060 23106 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 04:41:44.654269 23106 webserver.cc:460] Webserver started at http://127.0.0.1:38205/ using document root <none> and password file <none>
2026-08-11T04:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:44Z Disabled control of system clock
I20260811 04:41:44.664840 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35789
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:45465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:35789
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38441/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 04:41:44.779801 28952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:44.780093 28952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:44.780135 28952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:44.784194 28952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:44.784260 28952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:44.784276 28952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:44.784292 28952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:44.784307 28952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 04:41:44.784323 28952 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 04:41:44.784386 28952 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38441/jwks.json
I20260811 04:41:44.789150 28952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:35789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38441/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:44.790414 28952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:44.791879 28952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:44.800880 28957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:44.800877 28960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:44.801002 28952 server_base.cc:1034] running on GCE node
W20260811 04:41:44.800859 28958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:44.801855 28952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:44.803040 28952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:44.804219 28952 hybrid_clock.cc:648] HybridClock initialized: now 1786423304804176 us; error 55 us; skew 500 ppm
I20260811 04:41:44.806638 28952 webserver.cc:460] Webserver started at http://127.22.144.190:37703/ using document root <none> and password file <none>
I20260811 04:41:44.807242 28952 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:44.807310 28952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:44.807541 28952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:44.809813 28952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "56b340b05fb841d6a2cfe142330681aa"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "8ddb464bfd71e659825761a5b629e6b5"
server_key_iv: "9c52b9f90668c0eda7a320f1c235f0e7"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.810400 28952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "56b340b05fb841d6a2cfe142330681aa"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "8ddb464bfd71e659825761a5b629e6b5"
server_key_iv: "9c52b9f90668c0eda7a320f1c235f0e7"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.814342 28952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 04:41:44.816838 28966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:44.818212 28952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:44.818317 28952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "56b340b05fb841d6a2cfe142330681aa"
format_stamp: "Formatted at 2026-08-11 04:41:44 on dist-test-slave-ss4n"
server_key: "8ddb464bfd71e659825761a5b629e6b5"
server_key_iv: "9c52b9f90668c0eda7a320f1c235f0e7"
server_key_version: "encryptionkey@0"
I20260811 04:41:44.818413 28952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:44.829352 28952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:44.830096 28952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:44.830252 28952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:44.837759 28952 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:35789
I20260811 04:41:44.837775 29024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:35789 every 8 connection(s)
I20260811 04:41:44.838739 28952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:44.841349 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 28952
I20260811 04:41:44.841483 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:44.841884 23106 external_mini_cluster.cc:1428] Setting key a7f16c61d75bcc73a87d4b8f9c03cc9f
I20260811 04:41:44.842181 29025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:44.849232 29025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa: Bootstrap starting.
I20260811 04:41:44.851493 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:44.843361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.22.144.190:35789)
0811 04:41:44.843986 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:44.844002 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:44.844041 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:41:44.844841 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:44.844860 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:44.845273 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:44.845695 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:44.846121 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.847097 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.848006 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.848499 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.848686 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.849775 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:44.849780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:44.849793 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:44.850179 (+ 386us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:44.850238 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:44.850259 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:44.850376 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:44.850496 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 04:41:44.850766 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":146,"threads_started":1}
I20260811 04:41:44.852057 29025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:44.853186 29025 log.cc:824] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:44.854823 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:41:44.855793 29025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa: No bootstrap required, opened a new log
I20260811 04:41:44.860018 29025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b340b05fb841d6a2cfe142330681aa" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35789 } }
I20260811 04:41:44.860293 29025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:44.860394 29025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b340b05fb841d6a2cfe142330681aa, State: Initialized, Role: FOLLOWER
I20260811 04:41:44.860872 29025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b340b05fb841d6a2cfe142330681aa" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35789 } }
I20260811 04:41:44.861075 29025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:44.861178 29025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:44.861294 29025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:44.862630 29025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b340b05fb841d6a2cfe142330681aa" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35789 } }
I20260811 04:41:44.863057 29025 leader_election.cc:302] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56b340b05fb841d6a2cfe142330681aa; no voters:
I20260811 04:41:44.863374 29025 leader_election.cc:288] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:44.863567 29030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:44.863837 29030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [term 1 LEADER]: Becoming Leader. State: Replica: 56b340b05fb841d6a2cfe142330681aa, State: Running, Role: LEADER
I20260811 04:41:44.864197 29030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b340b05fb841d6a2cfe142330681aa" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35789 } }
I20260811 04:41:44.864733 29025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:44.866058 29032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56b340b05fb841d6a2cfe142330681aa. Latest consensus state: current_term: 1 leader_uuid: "56b340b05fb841d6a2cfe142330681aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b340b05fb841d6a2cfe142330681aa" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35789 } } }
I20260811 04:41:44.866226 29032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:44.866585 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56b340b05fb841d6a2cfe142330681aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56b340b05fb841d6a2cfe142330681aa" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35789 } } }
I20260811 04:41:44.866740 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:44.866664 29041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:44.871076 29041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:44.877228 29041 catalog_manager.cc:1347] Generated new cluster ID: fb294c32b42240f79d08d520cd439f81
I20260811 04:41:44.877312 29041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:44.897694 29041 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:44.898711 29041 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:44.906597 29041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56b340b05fb841d6a2cfe142330681aa: Generated new TSK 0
I20260811 04:41:44.907325 29041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:41:44.923882 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:44.918452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36012 (local address 127.22.144.190:35789)
0811 04:41:44.918585 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:44.918590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:44.918740 (+ 150us) server_negotiation.cc:407] Connection header received
0811 04:41:44.918891 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:44.918894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:44.918958 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:44.919081 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0811 04:41:44.919309 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.920129 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.921073 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.921407 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.921572 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.921946 (+ 374us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 04:41:44.923505 (+ 1559us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 04:41:44.923743 (+ 238us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36012: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 04:41:44.923960 29028 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36012: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 04:41:44.925583 29057 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:44.918350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:35789 (local address 127.0.0.1:36012)
0811 04:41:44.918632 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:44.918653 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:44.918757 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:44.919099 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:44.919101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:44.919108 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 04:41:44.919201 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:44.919206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.920299 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.920302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:44.920947 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:44.920955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:44.921663 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:44.921667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:44.921736 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:44.921765 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 04:41:44.923673 (+ 1908us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 04:41:44.925392 (+ 1719us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.144.190:35789: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":1529,"thread_start_us":118,"threads_started":1}
I20260811 04:41:44.926170 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 28952
E20260811 04:41:44.936012 29058 webserver.cc:533] Webserver: sslize failed:
E20260811 04:41:44.936084 29058 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T04:41:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (315 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T04:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:44Z Disabled control of system clock
I20260811 04:41:44.979410 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39825
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:39825
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/testchainca.cert with env {}
W20260811 04:41:45.097358 29063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:45.097718 29063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:45.097780 29063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:45.101779 29063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:45.101836 29063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:45.101852 29063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:45.101869 29063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:45.101884 29063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 04:41:45.101900 29063 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 04:41:45.101913 29063 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 04:41:45.107419 29063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:39825
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:45.108527 29063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:45.109668 29063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:45.118446 29069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:45.118435 29068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:45.118535 29063 server_base.cc:1034] running on GCE node
W20260811 04:41:45.118436 29071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:45.119308 29063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:45.120416 29063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:45.121592 29063 hybrid_clock.cc:648] HybridClock initialized: now 1786423305121552 us; error 46 us; skew 500 ppm
I20260811 04:41:45.123754 29063 webserver.cc:460] Webserver started at http://127.22.144.190:42605/ using document root <none> and password file <none>
I20260811 04:41:45.124367 29063 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:45.124464 29063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:45.124699 29063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:45.126876 29063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "7b211ca166334f3a94f52650fd4d03ac"
format_stamp: "Formatted at 2026-08-11 04:41:45 on dist-test-slave-ss4n"
server_key: "da4da905e97248f92847a443e402bc4b"
server_key_iv: "b308e3f44a779622bc1ed6276737449b"
server_key_version: "encryptionkey@0"
I20260811 04:41:45.127492 29063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "7b211ca166334f3a94f52650fd4d03ac"
format_stamp: "Formatted at 2026-08-11 04:41:45 on dist-test-slave-ss4n"
server_key: "da4da905e97248f92847a443e402bc4b"
server_key_iv: "b308e3f44a779622bc1ed6276737449b"
server_key_version: "encryptionkey@0"
I20260811 04:41:45.131367 29063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:45.134096 29077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:45.135519 29063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:45.135658 29063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "7b211ca166334f3a94f52650fd4d03ac"
format_stamp: "Formatted at 2026-08-11 04:41:45 on dist-test-slave-ss4n"
server_key: "da4da905e97248f92847a443e402bc4b"
server_key_iv: "b308e3f44a779622bc1ed6276737449b"
server_key_version: "encryptionkey@0"
I20260811 04:41:45.135783 29063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:45.147971 29063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:45.148729 29063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:45.148940 29063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:45.157343 29063 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:39825
I20260811 04:41:45.157872 29135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:39825 every 8 connection(s)
I20260811 04:41:45.158535 29063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:45.161623 29136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:45.165802 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29063
I20260811 04:41:45.165902 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:45.166203 23106 external_mini_cluster.cc:1428] Setting key f067832fc35862d3026d8e69ce289661
I20260811 04:41:45.168399 29136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac: Bootstrap starting.
I20260811 04:41:45.170895 29136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:45.171727 29136 log.cc:824] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:45.174065 29136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac: No bootstrap required, opened a new log
I20260811 04:41:45.176198 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:45.167679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.22.144.190:39825)
0811 04:41:45.168402 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:45.168419 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:45.168456 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:41:45.169128 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:45.169145 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:45.169707 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:45.170048 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:45.170507 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:45.171582 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:45.172737 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:45.173232 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:45.173403 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:45.174420 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:45.174425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:45.174438 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:41:45.174826 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:45.174884 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:41:45.174903 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:45.175019 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:45.175176 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 04:41:45.175434 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260811 04:41:45.178537 29136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b211ca166334f3a94f52650fd4d03ac" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39825 } }
I20260811 04:41:45.178800 29136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:45.178865 29136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b211ca166334f3a94f52650fd4d03ac, State: Initialized, Role: FOLLOWER
I20260811 04:41:45.179379 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:41:45.179409 29136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b211ca166334f3a94f52650fd4d03ac" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39825 } }
I20260811 04:41:45.179623 29136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:45.179721 29136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:45.179831 29136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:45.180975 29136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b211ca166334f3a94f52650fd4d03ac" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39825 } }
I20260811 04:41:45.181383 29136 leader_election.cc:302] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b211ca166334f3a94f52650fd4d03ac; no voters:
I20260811 04:41:45.181735 29136 leader_election.cc:288] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:45.181876 29141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:45.182109 29141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [term 1 LEADER]: Becoming Leader. State: Replica: 7b211ca166334f3a94f52650fd4d03ac, State: Running, Role: LEADER
I20260811 04:41:45.182427 29141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b211ca166334f3a94f52650fd4d03ac" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39825 } }
I20260811 04:41:45.183061 29136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:45.184178 29142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b211ca166334f3a94f52650fd4d03ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b211ca166334f3a94f52650fd4d03ac" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39825 } } }
I20260811 04:41:45.184358 29142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:45.184171 29143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b211ca166334f3a94f52650fd4d03ac. Latest consensus state: current_term: 1 leader_uuid: "7b211ca166334f3a94f52650fd4d03ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b211ca166334f3a94f52650fd4d03ac" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39825 } } }
I20260811 04:41:45.184598 29143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:45.184849 29150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:45.188218 29150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:45.194734 29150 catalog_manager.cc:1347] Generated new cluster ID: 0347cb904d524d319d57033de57b74b9
I20260811 04:41:45.194859 29150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:45.206777 29150 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:45.208117 29150 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:45.216040 29150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b211ca166334f3a94f52650fd4d03ac: Generated new TSK 0
I20260811 04:41:45.217065 29150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:41:46.496237 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:45.235391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.22.144.190:39825)
0811 04:41:45.235531 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:45.235537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:45.235796 (+ 259us) server_negotiation.cc:407] Connection header received
0811 04:41:45.236039 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:45.236042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:45.236098 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:45.236229 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0811 04:41:45.236505 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:45.237139 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:45.238197 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:45.238446 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:45.238582 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:45.238954 (+ 372us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 04:41:46.495629 (+1256675us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 04:41:46.496062 (+ 433us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36966: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 04:41:46.496348 29139 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36966: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 04:41:46.496394 29166 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:45.235301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39825 (local address 127.0.0.1:36966)
0811 04:41:45.235690 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:45.235703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:45.235857 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:45.236228 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:45.236232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:45.236241 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 04:41:45.236383 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:45.236391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:45.237273 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:45.237277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:45.238053 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:45.238062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:45.238631 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:45.238639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:45.238727 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:45.238752 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 04:41:46.495943 (+1257191us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 04:41:46.496183 (+ 240us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.144.190:39825: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":112,"threads_started":1}
I20260811 04:41:46.497383 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29063
2026-08-11T04:41:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1553 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (138 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (146 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:46 dist-test-slave-ss4n krb5kdc[29178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:46 dist-test-slave-ss4n krb5kdc[29178](info): set up 2 sockets
Aug 11 04:41:46 dist-test-slave-ss4n krb5kdc[29178](info): commencing operation
krb5kdc: starting...
W20260811 04:41:47.927138 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:48.076391 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:45849
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:45849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:48.198289 29192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:48.198575 29192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:48.198624 29192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:48.202703 29192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:48.202767 29192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:48.202787 29192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:48.202803 29192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:48.202819 29192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:48.208316 29192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:45849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:45849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:48.209518 29192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:48.210647 29192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:48.219905 29198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.220022 29199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:48.220005 29192 server_base.cc:1034] running on GCE node
W20260811 04:41:48.219905 29201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:48.220729 29192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:48.222008 29192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:48.223217 29192 hybrid_clock.cc:648] HybridClock initialized: now 1786423308223205 us; error 42 us; skew 500 ppm
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:48.226205 29192 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:41:48.227123 29192 webserver.cc:460] Webserver started at http://127.22.144.190:37041/ using document root <none> and password file <none>
I20260811 04:41:48.227645 29192 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:48.227703 29192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:48.227869 29192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:48.229954 29192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "e5584b83734c4137856f1ddc4573760e"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "85d7cbd90693eab1a7787f2aae9c7e79"
server_key_iv: "d1750e345612e701bda62c063542658f"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.230485 29192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "e5584b83734c4137856f1ddc4573760e"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "85d7cbd90693eab1a7787f2aae9c7e79"
server_key_iv: "d1750e345612e701bda62c063542658f"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.234306 29192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:41:48.236945 29209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.238548 29192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:41:48.238669 29192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "e5584b83734c4137856f1ddc4573760e"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "85d7cbd90693eab1a7787f2aae9c7e79"
server_key_iv: "d1750e345612e701bda62c063542658f"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.238781 29192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:48.252869 29192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:48.256418 29192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:48.256614 29192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:48.264570 29192 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:45849
I20260811 04:41:48.264617 29266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:45849 every 8 connection(s)
I20260811 04:41:48.265689 29192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:48.268954 29267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:48.274729 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29192
I20260811 04:41:48.274880 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:48.275190 23106 external_mini_cluster.cc:1428] Setting key affde1f32cb9c09b8d52550084b65453
I20260811 04:41:48.275439 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e: Bootstrap starting.
I20260811 04:41:48.278342 29267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:48.279196 29267 log.cc:824] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:48.281297 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e: No bootstrap required, opened a new log
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:48.285363 29267 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5584b83734c4137856f1ddc4573760e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45849 } }
I20260811 04:41:48.285742 29267 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:48.285840 29267 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5584b83734c4137856f1ddc4573760e, State: Initialized, Role: FOLLOWER
I20260811 04:41:48.286446 29267 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5584b83734c4137856f1ddc4573760e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45849 } }
I20260811 04:41:48.286634 29267 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:48.286741 29267 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:48.286854 29267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:48.287837 29267 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5584b83734c4137856f1ddc4573760e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45849 } }
I20260811 04:41:48.288220 29267 leader_election.cc:302] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5584b83734c4137856f1ddc4573760e; no voters:
I20260811 04:41:48.288569 29267 leader_election.cc:288] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:48.288802 29273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:48.289124 29273 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [term 1 LEADER]: Becoming Leader. State: Replica: e5584b83734c4137856f1ddc4573760e, State: Running, Role: LEADER
I20260811 04:41:48.289598 29273 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5584b83734c4137856f1ddc4573760e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45849 } }
I20260811 04:41:48.290239 29267 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:48.291359 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:48.276738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.22.144.190:45849)
0811 04:41:48.277274 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:48.277291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:48.277331 (+ 40us) server_negotiation.cc:407] Connection header received
0811 04:41:48.278142 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:48.278162 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:48.278584 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:48.278947 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:48.279832 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.280804 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.281750 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.282246 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.282418 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:48.284924 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:48.284930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:48.284943 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:48.284971 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:48.287296 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:48.287828 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:48.287834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:48.287841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:48.288003 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:48.288292 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:48.288295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:48.288297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:48.289150 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:48.289417 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:48.289906 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 04:41:48.290259 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":170,"threads_started":1}
I20260811 04:41:48.291541 29274 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5584b83734c4137856f1ddc4573760e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5584b83734c4137856f1ddc4573760e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45849 } } }
I20260811 04:41:48.291666 29274 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:48.292004 29275 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5584b83734c4137856f1ddc4573760e. Latest consensus state: current_term: 1 leader_uuid: "e5584b83734c4137856f1ddc4573760e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5584b83734c4137856f1ddc4573760e" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45849 } } }
I20260811 04:41:48.292122 29275 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e [sys.catalog]: This master's current role is: LEADER
W20260811 04:41:48.295388 29289 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:48.295460 29289 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:48.295548 29283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:48.296289 29283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:48.302006 29283 catalog_manager.cc:1347] Generated new cluster ID: 21db8f0c3ce24ff083c6e2816bad44f3
I20260811 04:41:48.302115 29283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:48.314334 29283 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:48.315586 29283 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:48.320958 29283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5584b83734c4137856f1ddc4573760e: Generated new TSK 0
I20260811 04:41:48.321771 29283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:41:48.357857 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45849
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:48.485085 29295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:48.485371 29295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:48.485421 29295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:48.490098 29295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:48.490168 29295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:48.490283 29295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:48.496587 29295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:48.497885 29295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:48.499107 29295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:48.509644 29300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.509677 29301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.509645 29303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:48.510085 29295 server_base.cc:1034] running on GCE node
I20260811 04:41:48.510509 29295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:48.511173 29295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:48.512374 29295 hybrid_clock.cc:648] HybridClock initialized: now 1786423308512353 us; error 36 us; skew 500 ppm
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:48.515746 29295 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:41:48.516849 29295 webserver.cc:460] Webserver started at http://127.22.144.129:35357/ using document root <none> and password file <none>
I20260811 04:41:48.517584 29295 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:48.517709 29295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:48.517997 29295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:48.520494 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "be6b85f7c0734cb5ac3aef5d42d6e068"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "20b0a8562d907a9db2cc25294a4fa57e"
server_key_iv: "07625b2fa154c463d8fed19002a5263e"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.521179 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "be6b85f7c0734cb5ac3aef5d42d6e068"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "20b0a8562d907a9db2cc25294a4fa57e"
server_key_iv: "07625b2fa154c463d8fed19002a5263e"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.525945 29295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:41:48.528795 29311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.530263 29295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:48.530418 29295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "be6b85f7c0734cb5ac3aef5d42d6e068"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "20b0a8562d907a9db2cc25294a4fa57e"
server_key_iv: "07625b2fa154c463d8fed19002a5263e"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.530555 29295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:48.542762 29295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:48.546020 29295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:48.546245 29295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:48.546852 29295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:48.547703 29295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:48.547777 29295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.547847 29295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:48.547899 29295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.557104 29295 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:41511
I20260811 04:41:48.557142 29435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:41511 every 8 connection(s)
I20260811 04:41:48.558089 29295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:48.566008 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29295
I20260811 04:41:48.566162 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:48.566563 23106 external_mini_cluster.cc:1428] Setting key 0a9a827c07ba50b798e60f0360658f54
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:48.574303 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:48.560210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:58803 (local address 127.22.144.190:45849)
0811 04:41:48.560361 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:48.560365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:48.561233 (+ 868us) server_negotiation.cc:407] Connection header received
0811 04:41:48.562772 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:48.562776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:48.562836 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:48.562949 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:48.564187 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.564735 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.566046 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.566435 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.566558 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:48.569651 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:48.569654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:48.569656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:48.569681 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:48.571502 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:48.572261 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:48.572267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:48.572269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:48.572373 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:48.572864 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:48.572870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:48.572873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:48.573154 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:48.573265 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:48.574062 (+ 797us) server_negotiation.cc:299] Negotiation successful
0811 04:41:48.574174 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:41:48.575642 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:48.560601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:45849 (local address 127.22.144.129:58803)
0811 04:41:48.561078 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:48.561121 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:48.562493 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:48.563137 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:48.563144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:48.563511 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:48.564015 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:48.564028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.564884 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.564888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:48.565900 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:48.565912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.566626 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.566631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:48.566805 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:48.567879 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:48.567900 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:48.569430 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:48.571642 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:48.571647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:48.571662 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:48.572107 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:48.572488 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:48.572494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:48.572499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:48.572741 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:48.573284 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:48.573294 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:48.573757 (+ 463us) client_negotiation.cc:769] Sending connection context
0811 04:41:48.573994 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 04:41:48.574396 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":115,"threads_started":1}
I20260811 04:41:48.577019 29436 heartbeater.cc:347] Connected to a master server at 127.22.144.190:45849
I20260811 04:41:48.577288 29436 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:48.577908 29436 heartbeater.cc:511] Master 127.22.144.190:45849 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:48.579542 29228 ts_manager.cc:194] Registered new tserver with Master: be6b85f7c0734cb5ac3aef5d42d6e068 (127.22.144.129:41511)
I20260811 04:41:48.581076 29228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:58803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:41:48.600984 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45849
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:48.727646 29441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:48.728039 29441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:48.728154 29441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:48.733094 29441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:48.733224 29441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:48.733332 29441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:48.739794 29441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:48.741123 29441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:48.742377 29441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:48.753028 29446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.753028 29447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.753044 29449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:48.753101 29441 server_base.cc:1034] running on GCE node
I20260811 04:41:48.753921 29441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:48.754704 29441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:48.755913 29441 hybrid_clock.cc:648] HybridClock initialized: now 1786423308755889 us; error 37 us; skew 500 ppm
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:48.759383 29441 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:41:48.760322 29441 webserver.cc:460] Webserver started at http://127.22.144.130:43445/ using document root <none> and password file <none>
I20260811 04:41:48.760982 29441 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:48.761049 29441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:48.761245 29441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:48.763453 29441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "e45c288ab6ac4fe0817d271fcf5e7839"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "1ddc6e946929be5a96d02b0e629985bd"
server_key_iv: "777877947c525bf91ef9542fc8fa35e0"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.764067 29441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "e45c288ab6ac4fe0817d271fcf5e7839"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "1ddc6e946929be5a96d02b0e629985bd"
server_key_iv: "777877947c525bf91ef9542fc8fa35e0"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.768154 29441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:48.770946 29456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.772233 29441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:48.772334 29441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "e45c288ab6ac4fe0817d271fcf5e7839"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "1ddc6e946929be5a96d02b0e629985bd"
server_key_iv: "777877947c525bf91ef9542fc8fa35e0"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.772544 29441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:48.792846 29441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:48.796244 29441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:48.796464 29441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:48.797339 29441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:48.798360 29441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:48.798411 29441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.798453 29441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:48.798508 29441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:48.807551 29441 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:41317
I20260811 04:41:48.807574 29580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:41317 every 8 connection(s)
I20260811 04:41:48.808539 29441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:48.809224 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29441
I20260811 04:41:48.809355 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:48.809764 23106 external_mini_cluster.cc:1428] Setting key 37f644be43039470bcfa012448b3af97
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:41:48.824790 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:48.810694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:36249 (local address 127.22.144.190:45849)
0811 04:41:48.810834 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:48.810839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:48.811849 (+ 1010us) server_negotiation.cc:407] Connection header received
0811 04:41:48.813277 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:48.813280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:48.813339 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:48.813443 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:48.814722 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.815284 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.817226 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.817653 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.817753 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:48.820851 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:48.820855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:48.820858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:48.820888 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:48.822432 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:48.822976 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:48.822981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:48.822983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:48.823076 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:48.823435 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:48.823440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:48.823441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:48.823640 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:48.823753 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:48.824345 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 04:41:48.824569 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:41:48.825677 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:48.811159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:45849 (local address 127.22.144.130:36249)
0811 04:41:48.811709 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:48.811752 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:48.813057 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:48.813680 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:48.813687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:48.814066 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:48.814558 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:48.814573 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.815436 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.815442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:48.817015 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:48.817029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:48.817838 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:48.817845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:48.818128 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:48.818850 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:48.818879 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:48.820678 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:48.822574 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:48.822580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:48.822596 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:48.822864 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:48.823183 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:48.823187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:48.823189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:48.823334 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:48.823752 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:48.823759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:48.824013 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 04:41:48.824223 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 04:41:48.824632 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":161,"threads_started":1}
I20260811 04:41:48.827070 29581 heartbeater.cc:347] Connected to a master server at 127.22.144.190:45849
I20260811 04:41:48.827369 29581 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:48.828063 29581 heartbeater.cc:511] Master 127.22.144.190:45849 requested a full tablet report, sending...
I20260811 04:41:48.829309 29228 ts_manager.cc:194] Registered new tserver with Master: e45c288ab6ac4fe0817d271fcf5e7839 (127.22.144.130:41317)
I20260811 04:41:48.830135 29228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:36249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:41:48.844645 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45849
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:45013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:48.962361 29587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:48.962664 29587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:48.962749 29587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:48.966873 29587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:48.966966 29587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:48.967065 29587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:48.972658 29587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45849
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:48.973877 29587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:48.974998 29587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:41:48.985291 29587 server_base.cc:1034] running on GCE node
W20260811 04:41:48.985236 29595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.985255 29596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:48.985239 29599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:48.986147 29587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:48.986892 29587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:48.988109 29587 hybrid_clock.cc:648] HybridClock initialized: now 1786423308988069 us; error 48 us; skew 500 ppm
Aug 11 04:41:48 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:48.991547 29587 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:41:48.992692 29587 webserver.cc:460] Webserver started at http://127.22.144.131:43583/ using document root <none> and password file <none>
I20260811 04:41:48.993355 29587 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:48.993518 29587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:48.993800 29587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:48.996141 29587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "273f9b84af1045c2b74178243c133f02"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "6ab716b20c41ea25d7357ffbc82b0c5d"
server_key_iv: "27a05dfc7669cfed58ad81632a4ecc1e"
server_key_version: "encryptionkey@0"
I20260811 04:41:48.996963 29587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "273f9b84af1045c2b74178243c133f02"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "6ab716b20c41ea25d7357ffbc82b0c5d"
server_key_iv: "27a05dfc7669cfed58ad81632a4ecc1e"
server_key_version: "encryptionkey@0"
I20260811 04:41:49.001387 29587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:49.004556 29606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:49.006137 29587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:49.006302 29587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "273f9b84af1045c2b74178243c133f02"
format_stamp: "Formatted at 2026-08-11 04:41:48 on dist-test-slave-ss4n"
server_key: "6ab716b20c41ea25d7357ffbc82b0c5d"
server_key_iv: "27a05dfc7669cfed58ad81632a4ecc1e"
server_key_version: "encryptionkey@0"
I20260811 04:41:49.006441 29587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:49.020712 29587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:49.024398 29587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:49.024649 29587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:49.025359 29587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:49.026350 29587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:49.026455 29587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:49.026525 29587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:49.026542 29587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:49.037200 29587 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:34053
I20260811 04:41:49.037271 29731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:34053 every 8 connection(s)
I20260811 04:41:49.038303 29587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:49.042738 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29587
I20260811 04:41:49.042922 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:49.043377 23106 external_mini_cluster.cc:1428] Setting key 409d3c98266bc00ffd1f55d1e2012677
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:41:49.054714 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:49.040740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:39439 (local address 127.22.144.190:45849)
0811 04:41:49.040931 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.040937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.042091 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 04:41:49.044057 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.044060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.044126 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.044257 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.045682 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.046277 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.047466 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.047702 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.047819 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:49.050587 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:49.050591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:49.050593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:49.050621 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:49.052345 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.052937 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.052940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.052942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.053024 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.053532 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.053537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.053539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.053752 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:49.053841 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:49.054455 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 04:41:49.054579 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 04:41:49.055680 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:49.041167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:45849 (local address 127.22.144.131:39439)
0811 04:41:49.041904 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:49.041973 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:49.043847 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:49.044461 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:49.044468 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:49.044857 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:49.045381 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:49.045396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.046412 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.046416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:49.047328 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:49.047338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.047865 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.047869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:49.048056 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:49.048662 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:49.048679 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:49.050386 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:49.052473 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:49.052479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:49.052493 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:49.052827 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:49.053171 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:49.053176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:49.053178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:49.053367 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:49.053892 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:49.053898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:49.054205 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 04:41:49.054387 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 04:41:49.054708 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":143,"threads_started":1}
I20260811 04:41:49.057057 29732 heartbeater.cc:347] Connected to a master server at 127.22.144.190:45849
I20260811 04:41:49.057355 29732 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:49.058022 29732 heartbeater.cc:511] Master 127.22.144.190:45849 requested a full tablet report, sending...
I20260811 04:41:49.059219 29228 ts_manager.cc:194] Registered new tserver with Master: 273f9b84af1045c2b74178243c133f02 (127.22.144.131:34053)
I20260811 04:41:49.059942 29228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:39439
I20260811 04:41:49.070235 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/test-admin.keytab.
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.089787 23106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 04:41:49.093189 29740 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:49.091423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.22.144.129:41511)
0811 04:41:49.091748 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.091756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.091785 (+ 29us) server_negotiation.cc:407] Connection header received
0811 04:41:49.091896 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.091910 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.092112 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.092272 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.093027 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40536: BlockingRecv error: recv got EOF from 127.0.0.1:40536 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
W20260811 04:41:49.093444 23106 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc' of type FILE
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.094765 23106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.095798 23106 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:49.107012 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:49.096795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.22.144.129:41511)
0811 04:41:49.096919 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.096922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.096986 (+ 64us) server_negotiation.cc:407] Connection header received
0811 04:41:49.097102 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.097105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.097146 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.097219 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.098070 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.098983 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.100019 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.100497 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.100611 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:49.103193 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:49.103203 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:49.103219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:49.103245 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:49.105059 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.105564 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.105570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.105571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.105654 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.105956 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.105960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.105961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.106225 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:49.106520 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:49.106697 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 04:41:49.106884 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 04:41:49.111284 29740 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:49.110150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.22.144.129:41511)
0811 04:41:49.110274 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.110281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.110335 (+ 54us) server_negotiation.cc:407] Connection header received
0811 04:41:49.110510 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.110513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.110576 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.110663 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.111170 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40554: BlockingRecv error: recv got EOF from 127.0.0.1:40554 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 04:41:49.111354 23106 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc' of type FILE
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.112406 23106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.113353 23106 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:49.123176 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:49.114294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.22.144.129:41511)
0811 04:41:49.114423 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.114427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.114530 (+ 103us) server_negotiation.cc:407] Connection header received
0811 04:41:49.114678 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.114682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.114723 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.114816 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.115346 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.115869 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.116692 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.116935 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.117040 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:49.119683 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:49.119687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:49.119689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:49.119714 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:49.121276 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.121886 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.121889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.121891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.121972 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.122270 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.122272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.122273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.122450 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:49.122584 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:49.122885 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 04:41:49.123041 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 04:41:49.126165 29740 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:49.125118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.22.144.129:41511)
0811 04:41:49.125271 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.125274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.125306 (+ 32us) server_negotiation.cc:407] Connection header received
0811 04:41:49.125435 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.125437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.125530 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.125618 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.126062 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40572: BlockingRecv error: recv got EOF from 127.0.0.1:40572 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 04:41:49.126287 23106 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786423274535345-23106-0/krb5kdc/krb5cc' of type FILE
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.127382 23106 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:49.128278 23106 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:41:49.137709 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:49.129112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.22.144.129:41511)
0811 04:41:49.129252 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:49.129256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:49.129370 (+ 114us) server_negotiation.cc:407] Connection header received
0811 04:41:49.129531 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:49.129534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:49.129576 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:49.129677 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:49.130212 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.130753 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.131612 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:49.131848 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:49.131955 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:49.134333 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:49.134339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:49.134346 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:49.134371 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:49.135909 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.136389 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.136393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.136395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.136482 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:49.136798 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:49.136801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:49.136803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:49.136989 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:49.137167 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:49.137393 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 04:41:49.137538 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:41:49.138736 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29295
I20260811 04:41:49.149025 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29441
I20260811 04:41:49.161705 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29587
I20260811 04:41:49.171255 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29192
2026-08-11T04:41:49Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2389 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29755](info): set up 2 sockets
Aug 11 04:41:49 dist-test-slave-ss4n krb5kdc[29755](info): commencing operation
krb5kdc: starting...
W20260811 04:41:50.290053 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:50 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab.
Entry for principal oryx/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab.
I20260811 04:41:50.433204 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42529
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab
--principal=oryx/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:50.551952 29770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:50.552304 29770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:50.552386 29770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:50.556576 29770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:50.556694 29770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:50.556732 29770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:50.556770 29770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:50.556806 29770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:50.562472 29770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab
--principal=oryx/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:50.563673 29770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:50.564755 29770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:50.573906 29775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:50.573895 29778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:50.574126 29770 server_base.cc:1034] running on GCE node
W20260811 04:41:50.573958 29776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:50.574997 29770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:50.576234 29770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:50.577432 29770 hybrid_clock.cc:648] HybridClock initialized: now 1786423310577366 us; error 80 us; skew 500 ppm
Aug 11 04:41:50 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:50.580374 29770 init.cc:377] Logged in from keytab as oryx/127.22.144.190@KRBTEST.COM (short username oryx)
I20260811 04:41:50.581323 29770 webserver.cc:460] Webserver started at http://127.22.144.190:33779/ using document root <none> and password file <none>
I20260811 04:41:50.581957 29770 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:50.582051 29770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:50.582314 29770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:50.584405 29770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "0c32601fed6840f1898841616a8a8815"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "27f174f1319efed635db0ba7f4eff667"
server_key_iv: "a06fa2883feef02b0481c747ebbd3a34"
server_key_version: "encryptionkey@0"
I20260811 04:41:50.584991 29770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "0c32601fed6840f1898841616a8a8815"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "27f174f1319efed635db0ba7f4eff667"
server_key_iv: "a06fa2883feef02b0481c747ebbd3a34"
server_key_version: "encryptionkey@0"
I20260811 04:41:50.588812 29770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:50.591521 29785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:50.592805 29770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 04:41:50.592962 29770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "0c32601fed6840f1898841616a8a8815"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "27f174f1319efed635db0ba7f4eff667"
server_key_iv: "a06fa2883feef02b0481c747ebbd3a34"
server_key_version: "encryptionkey@0"
I20260811 04:41:50.593088 29770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:50.615377 29770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:50.618907 29770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:50.619143 29770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:50.626809 29770 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:42529
I20260811 04:41:50.627128 29842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:42529 every 8 connection(s)
I20260811 04:41:50.627882 29770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:50.630746 29845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:50.631644 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29770
I20260811 04:41:50.631786 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:50.632102 23106 external_mini_cluster.cc:1428] Setting key 0ddb5edb1bb4d4fc1ff1218ddec5dc4d
I20260811 04:41:50.637115 29845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Bootstrap starting.
I20260811 04:41:50.639451 29845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:50.640301 29845 log.cc:824] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:41:50 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:50.642648 29845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: No bootstrap required, opened a new log
I20260811 04:41:50.647684 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:50.633695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.22.144.190:42529)
0811 04:41:50.634166 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:50.634181 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:50.634218 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:41:50.634926 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:50.634946 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:50.635295 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:50.635665 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:50.636538 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:50.637646 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:50.638554 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:50.639039 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:50.639226 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:50.642015 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:50.642021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:50.642039 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:50.642072 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:50.644257 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:50.644765 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:50.644772 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:50.644779 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:50.644946 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:50.645254 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:50.645258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:50.645261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:50.645828 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:50.646041 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:50.646321 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 04:41:50.646642 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":143,"threads_started":1}
I20260811 04:41:50.647557 29845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } }
I20260811 04:41:50.647814 29845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:50.647866 29845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c32601fed6840f1898841616a8a8815, State: Initialized, Role: FOLLOWER
I20260811 04:41:50.648356 29845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } }
I20260811 04:41:50.648514 29845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:50.648582 29845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:50.648675 29845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:50.649683 29845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } }
I20260811 04:41:50.650074 29845 leader_election.cc:302] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c32601fed6840f1898841616a8a8815; no voters:
I20260811 04:41:50.650408 29845 leader_election.cc:288] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:50.650640 29851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:50.650870 29851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 LEADER]: Becoming Leader. State: Replica: 0c32601fed6840f1898841616a8a8815, State: Running, Role: LEADER
I20260811 04:41:50.651188 29851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } }
I20260811 04:41:50.651962 29845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:50.653213 29853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } }
I20260811 04:41:50.653383 29853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:50.653797 29861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:50.654145 29854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c32601fed6840f1898841616a8a8815. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } }
I20260811 04:41:50.654255 29854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:50.656903 29861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:50.663568 29861 catalog_manager.cc:1347] Generated new cluster ID: 9e8412f884224e7b940d99127e9d9971
I20260811 04:41:50.663666 29861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:50.695017 29861 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:50.696384 29861 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:50.704434 29861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Generated new TSK 0
I20260811 04:41:50.705394 29861 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab.
Entry for principal oryx/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab.
I20260811 04:41:50.745512 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42529
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:50.864043 29873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:50.864344 29873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:50.864421 29873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:50.868479 29873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:50.868594 29873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:50.868700 29873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:50.874511 29873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:50.875595 29873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:50.876644 29873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:50.886721 29878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:50.886919 29873 server_base.cc:1034] running on GCE node
W20260811 04:41:50.886749 29881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:50.886747 29879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:50.887408 29873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:50.888065 29873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:50.889266 29873 hybrid_clock.cc:648] HybridClock initialized: now 1786423310889250 us; error 27 us; skew 500 ppm
Aug 11 04:41:50 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:50.892249 29873 init.cc:377] Logged in from keytab as oryx/127.22.144.129@KRBTEST.COM (short username oryx)
I20260811 04:41:50.893227 29873 webserver.cc:460] Webserver started at http://127.22.144.129:44263/ using document root <none> and password file <none>
I20260811 04:41:50.893875 29873 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:50.893970 29873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:50.894213 29873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:50.896324 29873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "9520c8dd2161b459532b503cf72e9095"
server_key_iv: "ca9d0274d6b0b4929e65dbe593f3d225"
server_key_version: "encryptionkey@0"
I20260811 04:41:50.896993 29873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "9520c8dd2161b459532b503cf72e9095"
server_key_iv: "ca9d0274d6b0b4929e65dbe593f3d225"
server_key_version: "encryptionkey@0"
I20260811 04:41:50.900971 29873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:50.903674 29889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:50.905131 29873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:50.905303 29873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "9520c8dd2161b459532b503cf72e9095"
server_key_iv: "ca9d0274d6b0b4929e65dbe593f3d225"
server_key_version: "encryptionkey@0"
I20260811 04:41:50.905447 29873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:50.925908 29873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:50.929582 29873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:50.929798 29873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:50.930440 29873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:50.931306 29873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:50.931377 29873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:50.931458 29873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:50.931501 29873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:50.940972 29873 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:39011
I20260811 04:41:50.940984 30013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:39011 every 8 connection(s)
I20260811 04:41:50.942055 29873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:50.944037 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 29873
I20260811 04:41:50.944169 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:50.944468 23106 external_mini_cluster.cc:1428] Setting key bf0ae2f70b4b9e7379017a16dd04babf
Aug 11 04:41:50 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:41:50.958850 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:50.943937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:33359 (local address 127.22.144.190:42529)
0811 04:41:50.944192 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:50.944198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:50.945095 (+ 897us) server_negotiation.cc:407] Connection header received
0811 04:41:50.946862 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:50.946864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:50.946908 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:50.946985 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:50.948717 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:50.949631 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:50.950814 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:50.951272 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:50.951408 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:50.954811 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:50.954815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:50.954816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:50.954841 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:50.956419 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:50.956934 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:50.956937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:50.956938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:50.957023 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:50.957429 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:50.957435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:50.957440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:50.957734 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:50.957818 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:50.958364 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 04:41:50.958477 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:50.959499 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:50.944365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:33359)
0811 04:41:50.944936 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:50.944981 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:50.946566 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:50.947214 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:50.947225 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:50.947780 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:50.948516 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:50.948543 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:50.949765 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:50.949768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:50.950680 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:50.950690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:50.951437 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:50.951440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:50.951667 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:50.952427 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:50.952451 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:50.954619 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:50.956532 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:50.956538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:50.956551 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:50.956828 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:50.957138 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:50.957141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:50.957142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:50.957326 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:50.957839 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:50.957844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:50.958112 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 04:41:50.958285 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 04:41:50.958557 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":155,"threads_started":1}
I20260811 04:41:50.960930 30015 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42529
I20260811 04:41:50.961221 30015 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:50.961869 30015 heartbeater.cc:511] Master 127.22.144.190:42529 requested a full tablet report, sending...
I20260811 04:41:50.963796 29800 ts_manager.cc:194] Registered new tserver with Master: ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011)
I20260811 04:41:50.965553 29800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:33359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab.
Entry for principal oryx/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab.
I20260811 04:41:50.980942 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42529
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:51.098505 30021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:51.098793 30021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:51.098838 30021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:51.102962 30021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:51.103031 30021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:51.103093 30021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:51.108640 30021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:51.109906 30021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:51.110999 30021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:51.121100 30031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.121124 30027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.121163 30026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:51.121542 30021 server_base.cc:1034] running on GCE node
I20260811 04:41:51.122252 30021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:51.122962 30021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:51.124173 30021 hybrid_clock.cc:648] HybridClock initialized: now 1786423311124160 us; error 29 us; skew 500 ppm
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:51.127550 30021 init.cc:377] Logged in from keytab as oryx/127.22.144.130@KRBTEST.COM (short username oryx)
I20260811 04:41:51.128504 30021 webserver.cc:460] Webserver started at http://127.22.144.130:41871/ using document root <none> and password file <none>
I20260811 04:41:51.129096 30021 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:51.129192 30021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:51.129455 30021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:51.131637 30021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "99979722e0764015a2801f38a06f7e31"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "3f1cd81726ff0b05a09e4b5b96bbfa87"
server_key_iv: "63adb05df9116e1c1a7e871bd9069b6f"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.132244 30021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "99979722e0764015a2801f38a06f7e31"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "3f1cd81726ff0b05a09e4b5b96bbfa87"
server_key_iv: "63adb05df9116e1c1a7e871bd9069b6f"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.136180 30021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 04:41:51.138903 30040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.140208 30021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:51.140352 30021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "99979722e0764015a2801f38a06f7e31"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "3f1cd81726ff0b05a09e4b5b96bbfa87"
server_key_iv: "63adb05df9116e1c1a7e871bd9069b6f"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.140466 30021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:51.158460 30021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:51.161885 30021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:51.162103 30021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:51.162971 30021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:51.163839 30021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:51.163910 30021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.163977 30021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:51.164014 30021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.172883 30021 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:43109
I20260811 04:41:51.172906 30164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:43109 every 8 connection(s)
I20260811 04:41:51.173890 30021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:51.179102 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 30021
I20260811 04:41:51.179231 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:51.179558 23106 external_mini_cluster.cc:1428] Setting key 1536f23d0cd5212f8ab46171bc91d0ad
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:51.191244 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.175806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:57417 (local address 127.22.144.190:42529)
0811 04:41:51.175965 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.175969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.176864 (+ 895us) server_negotiation.cc:407] Connection header received
0811 04:41:51.178581 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.178585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.178660 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.178758 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.180442 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.181282 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.183076 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.183296 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.183380 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.186492 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.186499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.186505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.186542 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.188567 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.189221 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.189226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.189230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.189329 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.189876 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.189882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.189886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.190169 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.190287 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.190888 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.191051 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":67}
WARNING: no policy specified for oryx/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:41:51.192006 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.176185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:57417)
0811 04:41:51.176704 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.176749 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.178309 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.178997 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.179008 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.179546 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.180263 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.180285 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.181430 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.181433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.182947 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.182963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.183457 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.183463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.183730 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.184406 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.184423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.186300 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.188718 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.188724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.188737 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.189074 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.189456 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.189459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.189461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.189749 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.190280 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.190285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.190596 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.190765 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.191038 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
I20260811 04:41:51.193316 30165 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42529
I20260811 04:41:51.193634 30165 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:51.194257 30165 heartbeater.cc:511] Master 127.22.144.190:42529 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.131@KRBTEST.COM" created.
I20260811 04:41:51.195456 29800 ts_manager.cc:194] Registered new tserver with Master: 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109)
I20260811 04:41:51.196345 29800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:57417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab.
Entry for principal oryx/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab.
I20260811 04:41:51.213811 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42529
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:51.335207 30171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:51.335508 30171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:51.335556 30171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:51.339589 30171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:51.339658 30171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:51.339720 30171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:51.345553 30171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:42529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:51.346707 30171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:51.347765 30171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:41:51.359017 30171 server_base.cc:1034] running on GCE node
W20260811 04:41:51.358973 30176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.358958 30182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.358956 30178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:51.359809 30171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:51.360503 30171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:51.361676 30171 hybrid_clock.cc:648] HybridClock initialized: now 1786423311361655 us; error 35 us; skew 500 ppm
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:51.364856 30171 init.cc:377] Logged in from keytab as oryx/127.22.144.131@KRBTEST.COM (short username oryx)
I20260811 04:41:51.365957 30171 webserver.cc:460] Webserver started at http://127.22.144.131:45521/ using document root <none> and password file <none>
I20260811 04:41:51.366568 30171 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:51.366643 30171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:51.366895 30171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:51.369159 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "f2542949f0b5440c9418f4fd18486d2d"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "fc2c9e501b1f91026e0f87a2e653be36"
server_key_iv: "4c63a53de34359c816f7c12f9c0328d2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.369838 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "f2542949f0b5440c9418f4fd18486d2d"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "fc2c9e501b1f91026e0f87a2e653be36"
server_key_iv: "4c63a53de34359c816f7c12f9c0328d2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.374084 30171 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:51.376775 30190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.378530 30171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:51.378742 30171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "f2542949f0b5440c9418f4fd18486d2d"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "fc2c9e501b1f91026e0f87a2e653be36"
server_key_iv: "4c63a53de34359c816f7c12f9c0328d2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.378880 30171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:51.400514 30171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:51.404059 30171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:51.404237 30171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:51.404779 30171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:51.405766 30171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:51.405815 30171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.405858 30171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:51.405874 30171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.415519 30312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:39975 every 8 connection(s)
I20260811 04:41:51.415525 30171 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:39975
I20260811 04:41:51.416545 30171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:51.422281 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 30171
I20260811 04:41:51.422390 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:51.422693 23106 external_mini_cluster.cc:1428] Setting key d606b47a3135bb284425ad88cc79941c
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:51.432011 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.418798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:47075 (local address 127.22.144.190:42529)
0811 04:41:51.418969 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.418974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.419996 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 04:41:51.421553 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.421557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.421628 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.421726 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.423223 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.423899 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.425598 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.425823 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.425915 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.428513 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.428516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.428518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.428544 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.430084 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.430636 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.430639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.430640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.430720 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.431023 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.431026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.431027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.431228 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.431307 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.431784 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.431887 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 04:41:51.432875 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.419193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:47075)
0811 04:41:51.419780 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.419827 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.421274 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.421927 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.421934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.422413 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.423052 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.423069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.424064 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.424068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.425414 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.425424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.425965 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.425968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.426118 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.426735 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.426752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.428333 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.430248 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.430253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.430266 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.430530 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.430798 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.430800 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.430801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.430941 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.431320 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.431325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.431572 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.431736 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.431997 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":109,"threads_started":1}
I20260811 04:41:51.434120 30314 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42529
I20260811 04:41:51.434381 30314 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:51.434903 30314 heartbeater.cc:511] Master 127.22.144.190:42529 requested a full tablet report, sending...
I20260811 04:41:51.436048 29800 ts_manager.cc:194] Registered new tserver with Master: f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975)
I20260811 04:41:51.436756 29800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:47075
I20260811 04:41:51.437124 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.144.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.189.keytab.
Entry for principal oryx/127.22.144.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.189.keytab.
I20260811 04:41:51.469414 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:37873
--webserver_interface=127.22.144.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.189.keytab
--principal=oryx/127.22.144.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42529,127.22.144.189:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:51.598644 30319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:51.598942 30319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:51.599035 30319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:51.603498 30319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:51.603560 30319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:51.603580 30319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:51.603597 30319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:51.603613 30319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:51.609175 30319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42529,127.22.144.189:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.189.keytab
--principal=oryx/127.22.144.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.144.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
W20260811 04:41:51.609509 30319 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.144.190:42529,127.22.144.189:37873'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 04:41:51.610507 30319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:51.611565 30319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:51.620707 30326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.620795 30324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.620687 30328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:51.621018 30319 server_base.cc:1034] running on GCE node
I20260811 04:41:51.621719 30319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:51.623090 30319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:51.624312 30319 hybrid_clock.cc:648] HybridClock initialized: now 1786423311624305 us; error 46 us; skew 500 ppm
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:51.627406 30319 init.cc:377] Logged in from keytab as oryx/127.22.144.189@KRBTEST.COM (short username oryx)
I20260811 04:41:51.628377 30319 webserver.cc:460] Webserver started at http://127.22.144.189:34397/ using document root <none> and password file <none>
I20260811 04:41:51.628942 30319 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:51.629014 30319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:51.629253 30319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:51.631783 30319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/instance:
uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "4e0f812945932a6141edbbb061db9755"
server_key_iv: "ecd44efba3936ed399f02b97fc188ef2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.632447 30319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal/instance:
uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "4e0f812945932a6141edbbb061db9755"
server_key_iv: "ecd44efba3936ed399f02b97fc188ef2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.636670 30319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:51.639289 30336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.640558 30319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:41:51.640681 30319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "4e0f812945932a6141edbbb061db9755"
server_key_iv: "ecd44efba3936ed399f02b97fc188ef2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.640789 30319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:51.655663 30319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:51.658931 30319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:51.659142 30319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:51.666807 30319 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.189:37873
I20260811 04:41:51.667125 30391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.189:37873 every 8 connection(s)
I20260811 04:41:51.667884 30319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
I20260811 04:41:51.670810 30392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:51.674800 30392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:41:51.677984 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 30319
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.189@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
I20260811 04:41:51.691570 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.677072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.22.144.190:42529)
0811 04:41:51.677223 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.677227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.678304 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 04:41:51.679564 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.679568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.679643 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.679740 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.681252 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.682152 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.683832 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.684077 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.684194 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.687164 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.687167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.687170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.687199 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.689056 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.689713 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.689716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.689718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.689811 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.690233 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.690236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.690238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.690555 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.690680 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.691236 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.691389 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:41:51.692369 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.677441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:48292)
0811 04:41:51.678115 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.678174 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.679314 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.679855 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.679864 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.680443 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.680971 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.680995 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.682318 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.682321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.683691 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.683709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.684244 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.684249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.684362 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.685137 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.685154 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.686956 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.689177 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.689185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.689194 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.689612 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.689915 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.689922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.689926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.690126 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.690685 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.690690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.690948 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.691146 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.691509 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"spinlock_wait_cycles":3456,"thread_start_us":145,"threads_started":1}
I20260811 04:41:51.692370 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.678935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.22.144.189:37873)
0811 04:41:51.679293 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.679301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.679336 (+ 35us) server_negotiation.cc:407] Connection header received
0811 04:41:51.679569 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.679575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.679811 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.679977 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.681015 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.682495 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.683684 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.684147 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.684334 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.687686 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.687696 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.687734 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.687763 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.689706 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.690233 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.690239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.690240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.690341 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.690618 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.690622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.690626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.690951 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.691145 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.691384 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.691572 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":254,"spinlock_wait_cycles":3840,"thread_start_us":80,"threads_started":1}
I20260811 04:41:51.693946 30392 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.189@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
I20260811 04:41:51.702647 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.694514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:35460)
0811 04:41:51.694665 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.694683 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.694776 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.695091 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.695094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.695259 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.695347 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.695353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.696138 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.696145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.696689 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.696697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.697748 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.697752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.697905 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.698390 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.698402 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.699619 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.701156 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.701162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.701165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.701347 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.701678 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.701681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.701683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.701761 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.702128 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.702131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.702230 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.702331 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.702474 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":35}
I20260811 04:41:51.702683 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.694664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.22.144.189:37873)
0811 04:41:51.694910 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.694932 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.694940 (+ 8us) server_negotiation.cc:407] Connection header received
0811 04:41:51.694962 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.694964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.694995 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.695077 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.695459 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.696000 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.696875 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.697635 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.697752 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.699762 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.699765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.699767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.699786 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.701051 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.701441 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.701447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.701449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.701582 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.701851 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.701856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.701857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.702028 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.702129 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.702389 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.702577 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":186}
I20260811 04:41:51.705785 30392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Bootstrap starting.
I20260811 04:41:51.708171 30392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:51.708985 30392 log.cc:824] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:51.711202 30392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: No bootstrap required, opened a new log
I20260811 04:41:51.714713 30392 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } }
I20260811 04:41:51.714956 30392 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:51.714994 30392 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e01731b6ca4c6caf87db0ce8453d4e, State: Initialized, Role: FOLLOWER
I20260811 04:41:51.715409 30392 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } }
I20260811 04:41:51.715910 30400 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } } }
I20260811 04:41:51.716033 30400 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:51.716728 30392 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 04:41:51.722093 30411 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:51.722214 30411 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:51.724160 30319 master_runner.cc:428] Detected that this master c3e01731b6ca4c6caf87db0ce8453d4e is joining an existing cluster
I20260811 04:41:51.724223 30319 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.144.189:37873
I20260811 04:41:51.735070 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.726896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.22.144.190:42529)
0811 04:41:51.727022 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.727027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.727042 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:51.727084 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.727087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.727134 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.727223 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.727843 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.728684 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.729751 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.730021 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.730124 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.731539 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.731546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.731551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.731590 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.733299 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.733803 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.733807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.733809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.733896 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.734193 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.734197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.734199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.734387 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.734497 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.734773 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.734894 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:41:51.735105 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.726870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.22.144.189:37873)
0811 04:41:51.726987 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.726991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.727659 (+ 668us) server_negotiation.cc:407] Connection header received
0811 04:41:51.727703 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.727705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.727746 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.727823 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.728279 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.728775 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.729996 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.730318 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.730455 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.731594 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.731597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.731600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.731623 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.733339 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.733919 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.733922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.733924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.734001 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.734248 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.734250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.734252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.734433 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.734499 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.734799 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.734923 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":44}
I20260811 04:41:51.735363 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.726468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:48302)
0811 04:41:51.726753 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.726770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.726894 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.727262 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.727265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.727448 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.727712 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.727719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.728803 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.728809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.729629 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.729640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.730162 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.730165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.730298 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.730867 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.730886 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.731387 (+ 501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.733418 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.733424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.733426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.733699 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.733999 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.734001 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.734003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.734098 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.734486 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.734489 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.734602 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.734713 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.735028 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":66,"thread_start_us":103,"threads_started":1}
I20260811 04:41:51.735702 30419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.726515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:35472)
0811 04:41:51.727448 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.727463 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.727571 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.727908 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.727910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.728076 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.728169 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.728174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.728869 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.728873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.729868 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.729879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.730428 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.730432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.730586 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.731027 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.731038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.731414 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.733470 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.733474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.733515 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.733816 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.734096 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.734099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.734100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.734169 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.734570 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.734573 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.734649 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.734853 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.734998 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":59,"thread_start_us":94,"threads_started":1}
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.190@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
I20260811 04:41:51.751231 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.740248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:35484)
0811 04:41:51.740664 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.740712 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.740899 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.741276 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.741283 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.741783 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.741913 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.741921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.742790 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.742795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.743526 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.743536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.744174 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.744177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.744267 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.745092 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.745112 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.747086 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.749137 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.749150 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.749159 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.749547 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.749850 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.749854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.749857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.749974 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.750614 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.750620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.750751 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.750888 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.751076 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260811 04:41:51.751276 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.740394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.22.144.189:37873)
0811 04:41:51.740535 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.740540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.740809 (+ 269us) server_negotiation.cc:407] Connection header received
0811 04:41:51.741071 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.741075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.741138 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.741235 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.742085 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.742655 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.743685 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.744045 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.744152 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.747239 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.747243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.747248 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.747278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.749006 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.749645 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.749649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.749651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.749742 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.750088 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.750092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.750094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.750322 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.750423 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.750984 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.751112 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 04:41:51.752804 29800 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.144.189:37873: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
member_type: NON_VOTER
last_known_addr {
host: "127.22.144.189"
port: 37873
}
attrs {
promote: true
}
}
dest_uuid: "0c32601fed6840f1898841616a8a8815"
cas_config_opid_index: -1
I20260811 04:41:51.753770 29800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } }
I20260811 04:41:51.755672 30424 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c3e01731b6ca4c6caf87db0ce8453d4e (127.22.144.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } }
I20260811 04:41:51.756418 30364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:51.757650 30364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Refusing update from remote peer 0c32601fed6840f1898841616a8a8815: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 04:41:51.757867 30424 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.144.189:37873
WARNING: no policy specified for oryx/127.22.144.188@KRBTEST.COM; defaulting to no policy
I20260811 04:41:51.758387 30424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:51.758757 30319 master.cc:537] Master@127.22.144.189:37873 shutting down...
I20260811 04:41:51.758659 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } }
I20260811 04:41:51.758823 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
W20260811 04:41:51.760828 29786 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 04:41:51.762092 29786 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 -> Peer c3e01731b6ca4c6caf87db0ce8453d4e (127.22.144.189:37873): Couldn't send request to peer c3e01731b6ca4c6caf87db0ce8453d4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.188@KRBTEST.COM" created.
I20260811 04:41:51.764633 30319 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:51.764945 30319 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:51.765074 30319 tablet_replica.cc:330] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: stopping tablet replica
I20260811 04:41:51.781271 30319 master.cc:552] Master@127.22.144.189:37873 shutdown complete.
I20260811 04:41:51.781363 30319 master_runner.cc:305] Clearing existing system tablet
I20260811 04:41:51.781808 30319 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:51.781890 30319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.188.keytab.
Entry for principal oryx/127.22.144.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.188.keytab.
I20260811 04:41:51.783717 30319 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:51.784595 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:40639
--webserver_interface=127.22.144.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.188.keytab
--principal=oryx/127.22.144.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:41:51.784694 30430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.784947 30319 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:51.785111 30319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:51.785158 30319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "4e0f812945932a6141edbbb061db9755"
server_key_iv: "ecd44efba3936ed399f02b97fc188ef2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.787621 30319 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:51.789660 30319 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 04:41:51.789755 30319 log.cc:1196] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 04:41:51.790014 30319 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Deleting consensus metadata
I20260811 04:41:51.790205 30319 master_runner.cc:315] Copying system tablet from 127.22.144.190:42529
I20260811 04:41:51.791621 30319 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: tablet copy: Beginning tablet copy session from remote peer at address 127.22.144.190:42529
I20260811 04:41:51.802232 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.792392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.22.144.190:42529)
0811 04:41:51.792537 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:51.792542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:51.792840 (+ 298us) server_negotiation.cc:407] Connection header received
0811 04:41:51.793046 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:51.793049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:51.793102 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:51.793156 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:51.793869 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.794798 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.796031 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.796480 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.796643 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.798159 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:51.798165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:51.798169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:51.798208 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:51.800230 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.800722 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.800726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.800728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.800819 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:51.801113 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:51.801116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:51.801118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:51.801410 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:51.801563 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:51.801901 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 04:41:51.802039 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:41:51.802922 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:51.792305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:48314)
0811 04:41:51.792690 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:51.792747 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:51.792927 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:51.793305 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:51.793308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:51.793574 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:51.793713 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.793721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.794943 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.794947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.795880 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:51.795891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:51.796625 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:51.796629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:51.796734 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:51.797379 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:51.797394 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:51.798000 (+ 606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:51.800348 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.800352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.800354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.800620 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.800917 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:51.800920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:51.800922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:51.801015 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:51.801612 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:51.801616 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:51.801719 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 04:41:51.802568 (+ 849us) client_negotiation.cc:240] Negotiation successful
0811 04:41:51.802726 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":106,"threads_started":1}
I20260811 04:41:51.803714 29825 tablet_copy_service.cc:140] P 0c32601fed6840f1898841616a8a8815: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c3e01731b6ca4c6caf87db0ce8453d4e ({username='oryx', principal='oryx/127.22.144.189@KRBTEST.COM'} at 127.0.0.1:48314)
I20260811 04:41:51.803910 29825 tablet_copy_service.cc:161] P 0c32601fed6840f1898841616a8a8815: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c3e01731b6ca4c6caf87db0ce8453d4e at {username='oryx', principal='oryx/127.22.144.189@KRBTEST.COM'} at 127.0.0.1:48314: session id = c3e01731b6ca4c6caf87db0ce8453d4e-00000000000000000000000000000000
I20260811 04:41:51.806226 29825 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 04:41:51.808346 30319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 04:41:51.811087 30319 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: tablet copy: Starting download of 0 data blocks...
I20260811 04:41:51.811304 30319 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: tablet copy: Starting download of 1 WAL segments...
I20260811 04:41:51.813463 30319 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 04:41:51.815678 29825 tablet_copy_service.cc:342] P 0c32601fed6840f1898841616a8a8815: Request end of tablet copy session c3e01731b6ca4c6caf87db0ce8453d4e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.144.189@KRBTEST.COM'} at 127.0.0.1:48314
I20260811 04:41:51.815894 29825 tablet_copy_service.cc:433] P 0c32601fed6840f1898841616a8a8815: ending tablet copy session c3e01731b6ca4c6caf87db0ce8453d4e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c3e01731b6ca4c6caf87db0ce8453d4e
I20260811 04:41:51.825165 30319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:51.827050 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.827282 30438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.827488 30439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:51.827559 30319 server_base.cc:1034] running on GCE node
I20260811 04:41:51.827808 30319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:51.828118 30319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:51.829270 30319 hybrid_clock.cc:648] HybridClock initialized: now 1786423311829256 us; error 34 us; skew 500 ppm
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:51.831162 30319 init.cc:377] Logged in from keytab as oryx/127.22.144.189@KRBTEST.COM (short username oryx)
I20260811 04:41:51.831696 30319 webserver.cc:460] Webserver started at http://127.22.144.189:42333/ using document root <none> and password file <none>
I20260811 04:41:51.831893 30319 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:51.831938 30319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:51.833418 30319 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:51.834539 30450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.834852 30319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:51.834929 30319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "4e0f812945932a6141edbbb061db9755"
server_key_iv: "ecd44efba3936ed399f02b97fc188ef2"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.835060 30319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:51.849319 30319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:51.851806 30319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:51.857211 30319 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.189:37873
I20260811 04:41:51.857427 30511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.189:37873 every 8 connection(s)
I20260811 04:41:51.857899 30319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
I20260811 04:41:51.859896 30512 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:41:51.861135 30512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Bootstrap starting.
I20260811 04:41:51.872496 30512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:41:51.872896 30512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: Bootstrap complete.
I20260811 04:41:51.873641 30512 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } }
I20260811 04:41:51.873805 30512 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c3e01731b6ca4c6caf87db0ce8453d4e, State: Initialized, Role: LEARNER
I20260811 04:41:51.873917 30512 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } }
I20260811 04:41:51.874473 30515 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } }
I20260811 04:41:51.874555 30512 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:51.874691 30515 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: LEARNER
I20260811 04:41:51.878404 30526 catalog_manager.cc:1259] Loaded cluster ID: 9e8412f884224e7b940d99127e9d9971
I20260811 04:41:51.878472 30526 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: loading cluster ID for follower catalog manager: success
I20260811 04:41:51.880650 30526 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: acquiring CA information for follower catalog manager: success
I20260811 04:41:51.882211 30526 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 04:41:51.939558 30431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:51.939898 30431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:51.940003 30431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:51.944427 30431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:51.944518 30431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:51.944561 30431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:51.944585 30431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:51.944617 30431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:51.950263 30431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.188.keytab
--principal=oryx/127.22.144.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:40639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.144.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:51.951354 30431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:51.952524 30431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:51.961922 30535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.962060 30533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:51.962113 30531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:51.962563 30431 server_base.cc:1034] running on GCE node
I20260811 04:41:51.963189 30431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:51.964413 30431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:51.965627 30431 hybrid_clock.cc:648] HybridClock initialized: now 1786423311965595 us; error 54 us; skew 500 ppm
Aug 11 04:41:51 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:51.968735 30015 heartbeater.cc:503] Master 127.22.144.190:42529 was elected leader, sending a full tablet report...
I20260811 04:41:51.969084 30431 init.cc:377] Logged in from keytab as oryx/127.22.144.188@KRBTEST.COM (short username oryx)
I20260811 04:41:51.970405 30431 webserver.cc:460] Webserver started at http://127.22.144.188:44979/ using document root <none> and password file <none>
I20260811 04:41:51.971076 30431 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:51.971145 30431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:51.971403 30431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:51.973909 30431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/instance:
uuid: "88c9f5c55e8c499cb9efa19599b946a2"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "1471c1bd0b0168fc5b4d0c839928c89e"
server_key_iv: "c49bae91b5fe3cef19c7887a57535b93"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.974680 30431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal/instance:
uuid: "88c9f5c55e8c499cb9efa19599b946a2"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "1471c1bd0b0168fc5b4d0c839928c89e"
server_key_iv: "c49bae91b5fe3cef19c7887a57535b93"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.979382 30431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:51.982175 30542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:51.983724 30431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:51.983860 30431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
uuid: "88c9f5c55e8c499cb9efa19599b946a2"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "1471c1bd0b0168fc5b4d0c839928c89e"
server_key_iv: "c49bae91b5fe3cef19c7887a57535b93"
server_key_version: "encryptionkey@0"
I20260811 04:41:51.983978 30431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:52.010249 30431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:52.014266 30431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:52.014513 30431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:52.025615 30431 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.188:40639
I20260811 04:41:52.025944 30599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.188:40639 every 8 connection(s)
I20260811 04:41:52.026887 30431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
I20260811 04:41:52.030273 30600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:52.034848 30600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:41:52.035992 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 30431
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.188@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.144.188@KRBTEST.COM
I20260811 04:41:52.051682 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.037813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.22.144.190:42529)
0811 04:41:52.038000 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.038008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.038497 (+ 489us) server_negotiation.cc:407] Connection header received
0811 04:41:52.039677 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.039684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.039755 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.039851 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.040962 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.041616 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.042897 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.043205 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.043308 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.047041 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.047049 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.047056 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.047102 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.048873 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.049635 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.049639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.049641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.049767 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.050140 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.050144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.050149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.050419 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.050529 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.051295 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.051464 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 04:41:52.052675 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.037660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.22.144.188:40639)
0811 04:41:52.038285 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.038305 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.038349 (+ 44us) server_negotiation.cc:407] Connection header received
0811 04:41:52.039368 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.039391 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.039629 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.039787 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.040680 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.041899 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.042975 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.043483 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.043726 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.047617 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.047630 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.047648 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.047687 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.049629 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.050176 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.050180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.050182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.050254 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.050568 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.050571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.050572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.051009 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.051257 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.051718 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.051856 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":160,"server-negotiator.queue_time_us":460,"thread_start_us":203,"threads_started":1}
I20260811 04:41:52.052732 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.037660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59476)
0811 04:41:52.038351 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.038394 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.039390 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.039947 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.039953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.040413 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.040797 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.040812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.041766 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.041770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.042740 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.042750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.043407 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.043412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.043665 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.044387 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.044404 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.046791 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.049035 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.049045 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.049056 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.049449 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.049871 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.049873 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.049874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.050026 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.050544 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.050633 (+ 89us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.051000 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.051186 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.051526 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":7,"spinlock_wait_cycles":14592,"thread_start_us":197,"threads_started":1}
I20260811 04:41:52.054296 30600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.188@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
I20260811 04:41:52.063305 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.054877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:46038)
0811 04:41:52.055057 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.055072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.055146 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.055515 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.055517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.055665 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.055780 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.055785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.056753 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.056756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.057291 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.057299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.057810 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.057812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.057889 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.058395 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.058408 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.060028 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.061886 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.061889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.061891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.062122 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.062372 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.062374 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.062375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.062451 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.062829 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.062831 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.062951 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.063062 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.063179 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 04:41:52.063419 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.054993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.22.144.189:37873)
0811 04:41:52.055244 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.055247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.055260 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:52.055327 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.055329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.055385 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.055479 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.055916 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.056639 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.057389 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.057709 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.057795 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.060231 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.060235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.060237 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.060262 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.061777 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.062214 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.062217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.062218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.062292 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.062528 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.062530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.062532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.062728 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.062806 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.063143 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.063250 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260811 04:41:52.064249 30600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423311, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.188@KRBTEST.COM for oryx/127.22.144.188@KRBTEST.COM
I20260811 04:41:52.072633 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.064834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40804)
0811 04:41:52.065027 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.065044 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.065161 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.065406 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.065409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.065714 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.065789 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.065795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.066544 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.066549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.067042 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.067049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.067537 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.067540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.067671 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.068100 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.068111 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.069577 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.071080 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.071085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.071087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.071276 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.071545 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.071549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.071551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.071632 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.072010 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.072013 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.072120 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.072225 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.072370 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":40}
I20260811 04:41:52.072633 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.064887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.22.144.188:40639)
0811 04:41:52.065028 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.065033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.065119 (+ 86us) server_negotiation.cc:407] Connection header received
0811 04:41:52.065258 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.065261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.065304 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.065395 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.065910 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.066437 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.067164 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.067421 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.067525 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.069712 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.069716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.069721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.069746 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.070968 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.071373 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.071378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.071379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.071456 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.071730 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.071733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.071735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.071906 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.072022 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.072281 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.072421 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":80}
I20260811 04:41:52.076862 30600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Bootstrap starting.
I20260811 04:41:52.079859 30600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:52.080854 30600 log.cc:824] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:52.083324 30600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: No bootstrap required, opened a new log
I20260811 04:41:52.087284 30600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } }
I20260811 04:41:52.087519 30600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:52.087620 30600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c9f5c55e8c499cb9efa19599b946a2, State: Initialized, Role: FOLLOWER
I20260811 04:41:52.088129 30600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } }
I20260811 04:41:52.088728 30606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } } }
I20260811 04:41:52.088878 30606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:52.089800 30600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 04:41:52.095264 30617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:41:52.095427 30617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:41:52.099383 30431 master_runner.cc:428] Detected that this master 88c9f5c55e8c499cb9efa19599b946a2 is joining an existing cluster
I20260811 04:41:52.099464 30431 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.144.188:40639
I20260811 04:41:52.106496 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29873
I20260811 04:41:52.128178 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:39011
--local_ip_for_outbound_sockets=127.22.144.129
--tserver_master_addrs=127.22.144.190:42529
--webserver_port=44263
--webserver_interface=127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:41:52.148147 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.101986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.22.144.190:42529)
0811 04:41:52.102134 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.102181 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.103212 (+ 1031us) server_negotiation.cc:407] Connection header received
0811 04:41:52.103432 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.103435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.103487 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.103540 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.104256 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.105078 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.114991 (+ 9913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.115563 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.115731 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.117915 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.117923 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.117929 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.117973 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.120747 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.121435 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.121441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.121446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.121587 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.146801 (+ 25214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.146810 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.146813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.147162 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.147261 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.147746 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.147927 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:41:52.148797 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.101995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59480)
0811 04:41:52.103110 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.103134 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.103308 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.103688 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.103691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.103925 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.104056 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.104064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.105250 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.105252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.114750 (+ 9498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.114776 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.115940 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.115948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.116113 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.116804 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.116829 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.117687 (+ 858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.120880 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.120885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.120889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.121266 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.121721 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.121726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.121731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.146603 (+ 24872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.147355 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.147360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.147493 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.148412 (+ 919us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.148594 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1046,"thread_start_us":181,"threads_started":1}
I20260811 04:41:52.150081 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.102741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40820)
0811 04:41:52.104070 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.104082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.104185 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.104450 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.104453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.104650 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.104748 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.104753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.105796 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.105798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.119377 (+ 13579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.119394 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.121020 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.121027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.121264 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.122205 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.122214 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.146148 (+ 23934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.148565 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.148567 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.148570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.148800 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.149066 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.149069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.149070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.149144 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.149581 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.149583 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.149667 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.149760 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.149894 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"thread_start_us":106,"threads_started":1}
I20260811 04:41:52.150946 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.103529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.22.144.188:40639)
0811 04:41:52.103634 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.103639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.104139 (+ 500us) server_negotiation.cc:407] Connection header received
0811 04:41:52.104268 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.104271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.104301 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.104342 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.104867 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.105652 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.119518 (+ 13866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.120020 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.120150 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.146363 (+ 26213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.146367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.146369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.146395 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.148436 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.148900 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.148904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.148907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.148984 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.149222 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.149227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.149229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.149423 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.149545 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.150686 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.150801 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 04:41:52.154816 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.101986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:46044)
0811 04:41:52.102380 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.102395 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.102991 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.103406 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.103408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.103655 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.103819 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.103825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.104899 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.104902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.106486 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.106496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.121995 (+ 15499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.121998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.122079 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.146264 (+ 24185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.146286 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.146940 (+ 654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.152909 (+ 5969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.152913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.152915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.153185 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.153647 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.153650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.153652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.153745 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.154203 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.154206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.154296 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.154393 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.154658 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":130,"threads_started":1}
I20260811 04:41:52.155431 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.102079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.22.144.189:37873)
0811 04:41:52.102271 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.102276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.102480 (+ 204us) server_negotiation.cc:407] Connection header received
0811 04:41:52.103136 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.103139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.103194 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.103249 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.103950 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.104780 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.106635 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.106991 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.107112 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.147467 (+ 40355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.147471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.147474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.147519 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.152731 (+ 5212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.153349 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.153353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.153355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.153531 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.153844 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.153847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.153849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.154066 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.154160 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.155142 (+ 982us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.155289 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.190@KRBTEST.COM for oryx/127.22.144.188@KRBTEST.COM
I20260811 04:41:52.165165 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.154614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40834)
0811 04:41:52.154870 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.154885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.154981 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.155752 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.155756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.155989 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.156108 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.156116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.157225 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.157228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.158084 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.158097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.158719 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.158723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.158821 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.159357 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.159372 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.160968 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.163273 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.163276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.163279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.163658 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.163987 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.163990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.163992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.164095 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.164605 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.164608 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.164721 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.164830 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.164995 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 04:41:52.165684 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.155126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.22.144.188:40639)
0811 04:41:52.155366 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.155370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.155385 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:52.155452 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.155456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.155511 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.155593 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.156250 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.157091 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.158219 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.158588 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.158706 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.161127 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.161132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.161135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.161164 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.163133 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.163775 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.163779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.163781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.163878 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.164201 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.164205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.164207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.164484 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.164605 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.164910 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.165049 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 04:41:52.167070 29800 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.144.188:40639: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2"
member_type: NON_VOTER
last_known_addr {
host: "127.22.144.188"
port: 40639
}
attrs {
promote: true
}
}
dest_uuid: "0c32601fed6840f1898841616a8a8815"
cas_config_opid_index: 5
I20260811 04:41:52.167738 29800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } }
I20260811 04:41:52.169216 30424 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.170899 30424 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.144.188:40639
I20260811 04:41:52.171087 30424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.171204 30424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.171802 30431 master.cc:537] Master@127.22.144.188:40639 shutting down...
I20260811 04:41:52.172128 30572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:52.173779 30572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Refusing update from remote peer 0c32601fed6840f1898841616a8a8815: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 04:41:52.174480 30423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 04:41:52.176785 29788 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 -> Peer 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188:40639): Couldn't send request to peer 88c9f5c55e8c499cb9efa19599b946a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 04:41:52.180013 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.169050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:46048)
0811 04:41:52.169380 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.169395 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.169682 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.170090 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.170093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.170110 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:52.170204 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.170210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.177053 (+ 6843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.177057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.178392 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.178406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.179599 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.179603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.179693 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.179740 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.179815 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.179896 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260811 04:41:52.180495 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.169158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.22.144.189:37873)
0811 04:41:52.169280 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.169287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.169606 (+ 319us) server_negotiation.cc:407] Connection header received
0811 04:41:52.169814 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.169817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.169880 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.169959 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:52.170325 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.171512 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.178564 (+ 7052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.179245 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.179369 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.179528 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.180291 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.180376 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 04:41:52.181782 30481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 LEARNER]: Refusing update from remote peer 0c32601fed6840f1898841616a8a8815: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 04:41:52.184768 30423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 04:41:52.185444 30431 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:52.185797 30431 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:52.185904 30431 tablet_replica.cc:330] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: stopping tablet replica
I20260811 04:41:52.187079 30481 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.189292 30631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c32601fed6840f1898841616a8a8815. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.189430 30631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: LEARNER
I20260811 04:41:52.189337 30515 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: NON_VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: true } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.189543 30515 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: LEARNER
I20260811 04:41:52.192575 30424 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: attempting to promote NON_VOTER c3e01731b6ca4c6caf87db0ce8453d4e to VOTER
I20260811 04:41:52.193107 30424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } }
I20260811 04:41:52.194512 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.194199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40848)
0811 04:41:52.194339 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.194418 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 04:41:52.194729 30481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 LEARNER]: Refusing update from remote peer 0c32601fed6840f1898841616a8a8815: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 04:41:52.195221 30425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 04:41:52.195639 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.195178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40858)
0811 04:41:52.195515 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.195566 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:40639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":99,"threads_started":1}
W20260811 04:41:52.195801 29788 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 -> Peer 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188:40639): Couldn't send request to peer 88c9f5c55e8c499cb9efa19599b946a2. Status: Network error: Client connection negotiation failed: client connection to 127.22.144.188:40639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 04:41:52.197134 30425 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c3e01731b6ca4c6caf87db0ce8453d4e (127.22.144.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.198874 30425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.198961 30165 heartbeater.cc:503] Master 127.22.144.190:42529 was elected leader, sending a full tablet report...
I20260811 04:41:52.199002 30425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.199128 30425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.199199 30425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.200011 30481 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c3e01731b6ca4c6caf87db0ce8453d4e (127.22.144.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.201714 30515 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.201833 30515 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:52.203500 30431 master.cc:552] Master@127.22.144.188:40639 shutdown complete.
I20260811 04:41:52.203572 30431 master_runner.cc:305] Clearing existing system tablet
I20260811 04:41:52.203821 30431 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:52.203889 30431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:52.205271 30431 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:52.206295 30634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.206558 30431 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:52.206848 30431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 04:41:52.206931 30431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
uuid: "88c9f5c55e8c499cb9efa19599b946a2"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "1471c1bd0b0168fc5b4d0c839928c89e"
server_key_iv: "c49bae91b5fe3cef19c7887a57535b93"
server_key_version: "encryptionkey@0"
I20260811 04:41:52.209084 30431 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:52.211565 30431 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 04:41:52.211740 30431 log.cc:1196] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 04:41:52.212064 30431 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Deleting consensus metadata
I20260811 04:41:52.212271 30431 master_runner.cc:315] Copying system tablet from 127.22.144.190:42529
I20260811 04:41:52.213598 30431 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: tablet copy: Beginning tablet copy session from remote peer at address 127.22.144.190:42529
I20260811 04:41:52.227331 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.217471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59494)
0811 04:41:52.217905 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.217920 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.218062 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.218380 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.218383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.218614 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.218751 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.218758 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.219842 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.219846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.220786 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.220796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.221398 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.221402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.221530 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.222639 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.222654 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.223134 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.225396 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.225400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.225402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.225715 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.226120 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.226123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.226125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.226226 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.226739 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.226742 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.226853 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.226964 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.227126 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":91,"threads_started":1}
I20260811 04:41:52.227346 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.217783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.22.144.190:42529)
0811 04:41:52.217924 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.217929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.217999 (+ 70us) server_negotiation.cc:407] Connection header received
0811 04:41:52.218177 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.218180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.218234 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.218347 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.218893 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.219713 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.220925 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.221264 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.221373 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.223307 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.223312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.223315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.223345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.225264 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.225833 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.225839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.225841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.225957 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.226348 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.226352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.226354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.226623 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.226734 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.227054 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.227188 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:41:52.228439 29825 tablet_copy_service.cc:140] P 0c32601fed6840f1898841616a8a8815: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 88c9f5c55e8c499cb9efa19599b946a2 ({username='oryx', principal='oryx/127.22.144.188@KRBTEST.COM'} at 127.0.0.1:59494)
I20260811 04:41:52.228572 29825 tablet_copy_service.cc:161] P 0c32601fed6840f1898841616a8a8815: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 88c9f5c55e8c499cb9efa19599b946a2 at {username='oryx', principal='oryx/127.22.144.188@KRBTEST.COM'} at 127.0.0.1:59494: session id = 88c9f5c55e8c499cb9efa19599b946a2-00000000000000000000000000000000
I20260811 04:41:52.229983 29825 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 04:41:52.231408 30431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 04:41:52.234010 30431 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: tablet copy: Starting download of 0 data blocks...
I20260811 04:41:52.234306 30431 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: tablet copy: Starting download of 1 WAL segments...
I20260811 04:41:52.236137 30431 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 04:41:52.238189 29825 tablet_copy_service.cc:342] P 0c32601fed6840f1898841616a8a8815: Request end of tablet copy session 88c9f5c55e8c499cb9efa19599b946a2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.144.188@KRBTEST.COM'} at 127.0.0.1:59494
I20260811 04:41:52.238288 29825 tablet_copy_service.cc:433] P 0c32601fed6840f1898841616a8a8815: ending tablet copy session 88c9f5c55e8c499cb9efa19599b946a2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 88c9f5c55e8c499cb9efa19599b946a2
I20260811 04:41:52.247689 30431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:52.249756 30646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.249845 30643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.250568 30644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:52.253939 30431 server_base.cc:1034] running on GCE node
I20260811 04:41:52.254160 30431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:52.254510 30431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:52.255666 30431 hybrid_clock.cc:648] HybridClock initialized: now 1786423312255653 us; error 34 us; skew 500 ppm
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:52.257431 30431 init.cc:377] Logged in from keytab as oryx/127.22.144.188@KRBTEST.COM (short username oryx)
I20260811 04:41:52.258060 30431 webserver.cc:460] Webserver started at http://127.22.144.188:34037/ using document root <none> and password file <none>
I20260811 04:41:52.258287 30431 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:52.258334 30431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:52.259804 30431 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 04:41:52.260867 30654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.261169 30431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:52.261245 30431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
uuid: "88c9f5c55e8c499cb9efa19599b946a2"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "1471c1bd0b0168fc5b4d0c839928c89e"
server_key_iv: "c49bae91b5fe3cef19c7887a57535b93"
server_key_version: "encryptionkey@0"
I20260811 04:41:52.261323 30431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260811 04:41:52.290561 30629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:52.290906 30629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:52.290966 30629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:52.296957 30629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:52.297052 30629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:52.297137 30629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:52.304973 30629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:39011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=44263
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:52.305464 30431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:52.306526 30629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:52.307561 30431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:52.307929 30629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:41:52.313115 30431 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.188:40639
I20260811 04:41:52.313972 30431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
I20260811 04:41:52.314004 30717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.188:40639 every 8 connection(s)
I20260811 04:41:52.315804 30718 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:41:52.317036 30718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Bootstrap starting.
W20260811 04:41:52.321516 30722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.321635 30721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.321755 30724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:52.322250 30629 server_base.cc:1034] running on GCE node
I20260811 04:41:52.322783 30629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:52.323684 30629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:52.324908 30629 hybrid_clock.cc:648] HybridClock initialized: now 1786423312324871 us; error 48 us; skew 500 ppm
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:52.328900 30629 init.cc:377] Logged in from keytab as oryx/127.22.144.129@KRBTEST.COM (short username oryx)
I20260811 04:41:52.330240 30629 webserver.cc:460] Webserver started at http://127.22.144.129:44263/ using document root <none> and password file <none>
I20260811 04:41:52.330924 30718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:41:52.331055 30629 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:52.331169 30629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:52.331280 30718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Bootstrap complete.
I20260811 04:41:52.331903 30718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } }
I20260811 04:41:52.332083 30718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88c9f5c55e8c499cb9efa19599b946a2, State: Initialized, Role: LEARNER
I20260811 04:41:52.332211 30718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } }
I20260811 04:41:52.332607 30732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.332744 30732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: This master's current role is: LEARNER
I20260811 04:41:52.332791 30718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:52.336052 30744 catalog_manager.cc:1259] Loaded cluster ID: 9e8412f884224e7b940d99127e9d9971
I20260811 04:41:52.336122 30744 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: loading cluster ID for follower catalog manager: success
I20260811 04:41:52.338222 30744 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: acquiring CA information for follower catalog manager: success
I20260811 04:41:52.338245 30629 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 04:41:52.339318 30744 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:41:52.340680 30746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.342149 30629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:52.342308 30629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a"
format_stamp: "Formatted at 2026-08-11 04:41:50 on dist-test-slave-ss4n"
server_key: "9520c8dd2161b459532b503cf72e9095"
server_key_iv: "ca9d0274d6b0b4929e65dbe593f3d225"
server_key_version: "encryptionkey@0"
I20260811 04:41:52.342942 30629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:52.360481 30629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:52.364228 30629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:52.364472 30629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:52.365183 30629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:52.366235 30629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:52.366287 30629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.366361 30629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:52.366395 30629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.379467 30629 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:39011
I20260811 04:41:52.379488 30878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:39011 every 8 connection(s)
I20260811 04:41:52.380645 30629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:52.389199 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 30629
I20260811 04:41:52.389422 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 30021
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.188@KRBTEST.COM
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
I20260811 04:41:52.405591 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.383541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:36011 (local address 127.22.144.189:37873)
0811 04:41:52.383988 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.383993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.385830 (+ 1837us) server_negotiation.cc:407] Connection header received
0811 04:41:52.386059 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.386063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.386123 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.386218 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.387368 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.388336 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.395350 (+ 7014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.395727 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.395831 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.400259 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.400264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.400267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.400301 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.402908 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.403750 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.403754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.403756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.403851 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.404283 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.404287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.404289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.404507 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.404626 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.405212 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.405356 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351}
I20260811 04:41:52.406210 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.383713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:40985 (local address 127.22.144.188:40639)
0811 04:41:52.384140 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.384154 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.385937 (+ 1783us) server_negotiation.cc:407] Connection header received
0811 04:41:52.386139 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.386143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.386191 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.386295 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.387342 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.388199 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.393768 (+ 5569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.394242 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.394338 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.400716 (+ 6378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.400726 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.400733 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.400779 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.402908 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.403753 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.403759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.403764 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.403865 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.404296 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.404299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.404301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.404568 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.404673 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.405866 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.406030 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":91,"threads_started":1}
I20260811 04:41:52.406838 30884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.383292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.22.144.129:36011)
0811 04:41:52.385728 (+ 2436us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.385745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.385927 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.386359 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.386363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.386704 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.387203 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.387213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.388425 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.388428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.392659 (+ 4231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.392679 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.396030 (+ 3351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.396035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.396202 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.398290 (+ 2088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.398310 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.400042 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.403066 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.403070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.403091 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.403618 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.403975 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.403979 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.403981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.404161 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.404627 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.404636 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.404932 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.405146 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.405987 (+ 841us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2334,"mutex_wait_us":2468,"spinlock_wait_cycles":60672,"thread_start_us":155,"threads_started":1}
I20260811 04:41:52.406842 30883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.383292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.22.144.129:40985)
0811 04:41:52.385838 (+ 2546us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.385854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.386004 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.386311 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.386314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.386671 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.387186 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.387200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.388365 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.388369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.393609 (+ 5240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.393625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.394580 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.394586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.394889 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.396527 (+ 1638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.396555 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.400521 (+ 3966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.403059 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.403070 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.403092 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.403613 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.403983 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.403988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.403993 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.404175 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.404739 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.404745 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.404934 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.405148 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.405755 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2460,"mutex_wait_us":487,"spinlock_wait_cycles":23424,"thread_start_us":144,"threads_started":1}
I20260811 04:41:52.408339 30881 heartbeater.cc:347] Connected to a master server at 127.22.144.189:37873
I20260811 04:41:52.408615 30881 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.409164 30881 heartbeater.cc:511] Master 127.22.144.189:37873 requested a full tablet report, sending...
I20260811 04:41:52.409756 30879 heartbeater.cc:347] Connected to a master server at 127.22.144.188:40639
I20260811 04:41:52.409865 30879 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.410075 30879 heartbeater.cc:511] Master 127.22.144.188:40639 requested a full tablet report, sending...
I20260811 04:41:52.410904 30463 ts_manager.cc:194] Registered new tserver with Master: ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011)
I20260811 04:41:52.411291 30677 ts_manager.cc:194] Registered new tserver with Master: ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011)
I20260811 04:41:52.411741 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:43109
--local_ip_for_outbound_sockets=127.22.144.130
--tserver_master_addrs=127.22.144.190:42529
--webserver_port=41871
--webserver_interface=127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:52.420562 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.382788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:54615 (local address 127.22.144.190:42529)
0811 04:41:52.382971 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.382978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.384633 (+ 1655us) server_negotiation.cc:407] Connection header received
0811 04:41:52.385631 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.385634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.385689 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.385806 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.387369 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.388288 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.413852 (+ 25564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.414111 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.414249 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.417050 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.417053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.417056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.417085 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.418787 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.419261 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.419265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.419267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.419365 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.419661 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.419665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.419667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.419874 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.419989 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.420270 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.420393 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 04:41:52.420581 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.383292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:54615)
0811 04:41:52.384465 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.384509 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.385368 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.386038 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.386048 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.386713 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.387204 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.387213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.388456 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.388459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.413706 (+ 25247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.413719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.414258 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.414262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.414356 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.415488 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.415500 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.416865 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.418925 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.418929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.418931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.419152 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.419470 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.419473 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.419475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.419563 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.419980 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.419983 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.420095 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.420206 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.420380 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":24210,"spinlock_wait_cycles":1408,"thread_start_us":148,"threads_started":1}
I20260811 04:41:52.421654 30880 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42529
I20260811 04:41:52.421788 30880 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.421989 30880 heartbeater.cc:511] Master 127.22.144.190:42529 requested a full tablet report, sending...
I20260811 04:41:52.422878 29800 ts_manager.cc:194] Re-registered known tserver with Master: ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011)
I20260811 04:41:52.423498 29800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:54615
I20260811 04:41:52.439026 30314 heartbeater.cc:503] Master 127.22.144.190:42529 was elected leader, sending a full tablet report...
W20260811 04:41:52.532548 30888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:52.532876 30888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:52.532923 30888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:52.537451 30888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:52.537551 30888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:52.537619 30888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:52.543258 30888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:43109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=41871
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:52.544554 30888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:52.545717 30888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:52.556792 30894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.556805 30895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:52.556895 30888 server_base.cc:1034] running on GCE node
W20260811 04:41:52.556936 30897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:52.557773 30888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:52.558508 30888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:52.559701 30888 hybrid_clock.cc:648] HybridClock initialized: now 1786423312559669 us; error 56 us; skew 500 ppm
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:52.562915 30888 init.cc:377] Logged in from keytab as oryx/127.22.144.130@KRBTEST.COM (short username oryx)
I20260811 04:41:52.563866 30888 webserver.cc:460] Webserver started at http://127.22.144.130:41871/ using document root <none> and password file <none>
I20260811 04:41:52.564420 30888 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:52.564500 30888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:52.568696 30888 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:52.570959 30904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.572489 30888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:52.572608 30888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "99979722e0764015a2801f38a06f7e31"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "3f1cd81726ff0b05a09e4b5b96bbfa87"
server_key_iv: "63adb05df9116e1c1a7e871bd9069b6f"
server_key_version: "encryptionkey@0"
I20260811 04:41:52.573297 30888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:52.584246 30888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:52.587430 30888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:52.587599 30888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:52.588142 30888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:52.588997 30888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:52.589047 30888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.589089 30888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:52.589105 30888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.599532 30888 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:43109
I20260811 04:41:52.599551 31040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:43109 every 8 connection(s)
I20260811 04:41:52.600649 30888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:52.609876 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 30888
I20260811 04:41:52.610105 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 30171
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:52.620255 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.602582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:50953 (local address 127.22.144.190:42529)
0811 04:41:52.602742 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.602749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.604412 (+ 1663us) server_negotiation.cc:407] Connection header received
0811 04:41:52.605372 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.605375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.605431 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.605512 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.606947 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.607750 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609169 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.609399 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609458 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.616439 (+ 6981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.616443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.616445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.616474 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.618087 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.618632 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.618636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.618638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.618722 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.619093 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.619097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.619099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.619299 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.619405 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.619949 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.620091 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:41:52.621207 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.603191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:50953)
0811 04:41:52.604324 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.604337 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.605166 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.605724 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.605731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.606216 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.606782 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.606801 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.607896 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.607900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.609018 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.609029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.609740 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.609906 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.614577 (+ 4671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.614590 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.616231 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.618217 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.618223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.618242 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.618519 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.618827 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.618830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.618832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.618989 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.619419 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.619428 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.619709 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.619907 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.620259 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":392,"spinlock_wait_cycles":9984,"thread_start_us":111,"threads_started":1}
I20260811 04:41:52.622444 31042 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42529
I20260811 04:41:52.622718 31042 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.623310 31042 heartbeater.cc:511] Master 127.22.144.190:42529 requested a full tablet report, sending...
I20260811 04:41:52.624529 29800 ts_manager.cc:194] Re-registered known tserver with Master: 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109)
I20260811 04:41:52.625144 29800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:50953
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.188@KRBTEST.COM
I20260811 04:41:52.630553 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.603095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:51819 (local address 127.22.144.188:40639)
0811 04:41:52.603219 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.603223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.604413 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 04:41:52.605372 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.605375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.605433 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.605508 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.606947 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.607750 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609155 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.609526 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609655 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.626646 (+ 16991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.626652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.626656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.626696 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.628678 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.629208 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.629212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.629214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.629308 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.629621 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.629624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.629626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.629841 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.629939 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.630253 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.630398 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:41:52.630594 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.603191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.22.144.130:51819)
0811 04:41:52.604271 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.604314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.605166 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.605724 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.605729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.606223 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.606782 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.606801 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.607896 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.607900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.609045 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.609054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.609811 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.610002 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.624670 (+ 14668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.624688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.626444 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.628805 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.628808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.628810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.629050 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.629407 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.629410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.629412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.629531 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.629937 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.629940 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.630032 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.630156 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.630305 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":244,"spinlock_wait_cycles":120960,"thread_start_us":139,"threads_started":1}
I20260811 04:41:52.633002 31041 heartbeater.cc:347] Connected to a master server at 127.22.144.188:40639
I20260811 04:41:52.633121 31041 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.633368 31041 heartbeater.cc:511] Master 127.22.144.188:40639 requested a full tablet report, sending...
I20260811 04:41:52.634287 30677 ts_manager.cc:194] Registered new tserver with Master: 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109)
I20260811 04:41:52.636430 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:39975
--local_ip_for_outbound_sockets=127.22.144.131
--tserver_master_addrs=127.22.144.190:42529
--webserver_port=45521
--webserver_interface=127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
I20260811 04:41:52.641216 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.602566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:41665 (local address 127.22.144.189:37873)
0811 04:41:52.602704 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.602708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.604421 (+ 1713us) server_negotiation.cc:407] Connection header received
0811 04:41:52.605359 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.605363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.605424 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.605554 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.606927 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.607455 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.608625 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.608851 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.608989 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.637845 (+ 28856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.637850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.637854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.637888 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.639485 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.639959 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.639963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.639965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.640048 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.640351 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.640355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.640357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.640561 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.640672 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.640933 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.641052 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:41:52.641237 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.603191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.22.144.130:41665)
0811 04:41:52.604269 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.604311 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.605148 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.605723 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.605729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.606190 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.606781 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.606796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.607616 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.607619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.608495 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.608505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.609170 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.609175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.609361 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.610097 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.610120 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.637654 (+ 27534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.639633 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.639637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.639639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.639852 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.640166 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.640169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.640171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.640258 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.640662 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.640666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.640772 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.640881 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.641055 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":89,"spinlock_wait_cycles":81280,"thread_start_us":169,"threads_started":1}
I20260811 04:41:52.642064 31043 heartbeater.cc:347] Connected to a master server at 127.22.144.189:37873
I20260811 04:41:52.642203 31043 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.642418 31043 heartbeater.cc:511] Master 127.22.144.189:37873 requested a full tablet report, sending...
I20260811 04:41:52.643246 30463 ts_manager.cc:194] Registered new tserver with Master: 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109)
I20260811 04:41:52.678839 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.674822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.22.144.188:40639)
0811 04:41:52.675000 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.675004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.675022 (+ 18us) server_negotiation.cc:407] Connection header received
0811 04:41:52.675160 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.675164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.675229 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.675349 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:52.675634 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.676343 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.677615 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.678231 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.678370 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.678535 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.678619 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.678690 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 04:41:52.678839 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.674705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40874)
0811 04:41:52.674859 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.674877 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.675038 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.675368 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.675372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.675393 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:52.675514 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.675519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.676469 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.676473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.677407 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.677419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.678370 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.678374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.678472 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.678498 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.678606 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.678696 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 04:41:52.679675 30688 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 04:41:52.680112 30732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c32601fed6840f1898841616a8a8815. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.680276 30732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: This master's current role is: LEARNER
W20260811 04:41:52.764267 31049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:52.764616 31049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:52.764736 31049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:52.769443 31049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:52.769632 31049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:52.769735 31049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:52.776768 31049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:39975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=45521
--tserver_master_addrs=127.22.144.190:42529,127.22.144.189:37873,127.22.144.188:40639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:52.778158 31049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:52.779317 31049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:52.789968 31055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.789988 31058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:52.790010 31056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:52.790361 31049 server_base.cc:1034] running on GCE node
I20260811 04:41:52.790794 31049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:52.791533 31049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:52.792717 31049 hybrid_clock.cc:648] HybridClock initialized: now 1786423312792696 us; error 30 us; skew 500 ppm
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:52.795766 31049 init.cc:377] Logged in from keytab as oryx/127.22.144.131@KRBTEST.COM (short username oryx)
I20260811 04:41:52.796852 31049 webserver.cc:460] Webserver started at http://127.22.144.131:45521/ using document root <none> and password file <none>
I20260811 04:41:52.797472 31049 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:52.797655 31049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:52.802100 31049 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 04:41:52.804591 31067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.806052 31049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:52.806196 31049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "f2542949f0b5440c9418f4fd18486d2d"
format_stamp: "Formatted at 2026-08-11 04:41:51 on dist-test-slave-ss4n"
server_key: "fc2c9e501b1f91026e0f87a2e653be36"
server_key_iv: "4c63a53de34359c816f7c12f9c0328d2"
server_key_version: "encryptionkey@0"
I20260811 04:41:52.806772 31049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:52.821682 31049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:52.825546 31049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:52.825798 31049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:52.826505 31049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:41:52.827462 31049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:52.827538 31049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.827615 31049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:52.827668 31049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:52.839951 31049 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:39975
I20260811 04:41:52.839993 31196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:39975 every 8 connection(s)
I20260811 04:41:52.841408 31049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:52.844761 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 31049
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.189@KRBTEST.COM
Aug 11 04:41:52 dist-test-slave-ss4n krb5kdc[29755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423312, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.188@KRBTEST.COM
I20260811 04:41:52.859578 31212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.847524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.22.144.189:37873)
0811 04:41:52.847794 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.847798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.849018 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 04:41:52.849059 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.849062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.849104 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.849181 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.850265 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.851068 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.852268 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.852592 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.852723 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.854525 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.854531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.854535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.854567 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.856672 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.857749 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.857753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.857754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.857850 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.858178 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.858181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.858185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.858432 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.858568 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.859262 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.859397 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260811 04:41:52.862825 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.843507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:48037 (local address 127.22.144.190:42529)
0811 04:41:52.843682 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.843687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.845596 (+ 1909us) server_negotiation.cc:407] Connection header received
0811 04:41:52.846321 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.846326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.846405 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.846539 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.848580 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.849378 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.851554 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.851901 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.851997 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.857439 (+ 5442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.857443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.857445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.857472 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.859456 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.860360 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.860364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.860366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.860919 (+ 553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.861406 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.861410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.861414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.861665 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.861759 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.862531 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.862661 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 04:41:52.864380 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.847018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.22.144.188:40639)
0811 04:41:52.852031 (+ 5013us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.852037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.852053 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:52.852101 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.852105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.852168 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.852267 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.855264 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.856135 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.857342 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.857756 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.857850 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.859470 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.859474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.859477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.859515 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.861628 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.862143 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.862148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.862153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.862262 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.862596 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.862599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.862601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.863165 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.863304 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.863964 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.864102 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4927,"thread_start_us":86,"threads_started":1}
I20260811 04:41:52.864415 31204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.843931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:48037)
0811 04:41:52.845456 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.845474 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.846045 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.846743 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.846757 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.847489 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.848319 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.848345 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.849586 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.849591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.851412 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.851423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.852189 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.852197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.852461 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.853534 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.853550 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.857235 (+ 3685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.859605 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.859610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.859627 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.859975 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.861038 (+ 1063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.861041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.861043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.861294 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.861827 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.861833 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.862190 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.862986 (+ 796us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.863371 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1421,"mutex_wait_us":249,"spinlock_wait_cycles":2304,"thread_start_us":168,"threads_started":1}
I20260811 04:41:52.867693 31199 heartbeater.cc:347] Connected to a master server at 127.22.144.190:42529
I20260811 04:41:52.867972 31199 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.868592 31199 heartbeater.cc:511] Master 127.22.144.190:42529 requested a full tablet report, sending...
I20260811 04:41:52.869850 29800 ts_manager.cc:194] Re-registered known tserver with Master: f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975)
I20260811 04:41:52.870546 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.845742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:60517 (local address 127.22.144.189:37873)
0811 04:41:52.846457 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.846462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.846476 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:52.846520 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.846523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.846570 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.846660 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.848542 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.849529 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.851719 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.852114 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.852218 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.866400 (+ 14182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.866403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.866406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.866434 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.868339 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.869067 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.869070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.869072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.869169 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.869545 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.869549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.869550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.869781 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.869855 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.870271 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.870391 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619}
I20260811 04:41:52.870612 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.845580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:33397 (local address 127.22.144.188:40639)
0811 04:41:52.846886 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.846893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.846909 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:52.846956 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.846958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.847006 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.847080 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.848543 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.849517 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.853524 (+ 4007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.853992 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.854142 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.866009 (+ 11867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.866013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.866016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.866042 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.868481 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.869106 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.869109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.869111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.869208 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.869567 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.869571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.869572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.869774 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.869852 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.870361 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.870472 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1179}
I20260811 04:41:52.870636 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.843931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.22.144.131:33397)
0811 04:41:52.844530 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.844591 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.846341 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.848018 (+ 1677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.848021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.848216 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.848354 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.848362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.849994 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.849998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.853355 (+ 3357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.853369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.854357 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.854365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.854678 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.855239 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.855255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.865849 (+ 10594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.868615 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.868621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.868625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.868987 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.869317 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.869320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.869322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.869399 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.870087 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.870090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.870182 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.870304 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.870461 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":131,"thread_start_us":628,"threads_started":1}
I20260811 04:41:52.870632 29800 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:48037
I20260811 04:41:52.871192 31197 heartbeater.cc:347] Connected to a master server at 127.22.144.188:40639
I20260811 04:41:52.871245 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.845148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.22.144.131:60517)
0811 04:41:52.845980 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.845996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.846093 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.847298 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.847303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.847602 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:52.848316 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.848343 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.849703 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.849707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.851542 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.851558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.852567 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.852572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.852750 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.853383 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:52.853423 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:52.866229 (+ 12806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:52.868544 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.868548 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.868550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.868947 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.869292 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:52.869295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:52.869297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:52.869394 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:52.869993 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:52.869996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:52.870085 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.870909 (+ 824us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.871060 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":223,"spinlock_wait_cycles":4992,"thread_start_us":121,"threads_started":1}
I20260811 04:41:52.871279 31197 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.871476 31197 heartbeater.cc:511] Master 127.22.144.188:40639 requested a full tablet report, sending...
I20260811 04:41:52.871840 31200 heartbeater.cc:347] Connected to a master server at 127.22.144.189:37873
I20260811 04:41:52.871927 31200 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:52.872128 31200 heartbeater.cc:511] Master 127.22.144.189:37873 requested a full tablet report, sending...
I20260811 04:41:52.872254 30677 ts_manager.cc:194] Registered new tserver with Master: f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975)
I20260811 04:41:52.872848 30463 ts_manager.cc:194] Registered new tserver with Master: f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975)
I20260811 04:41:52.875586 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.849720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59506 (local address 127.22.144.190:42529)
0811 04:41:52.851274 (+ 1554us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.851295 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.856475 (+ 5180us) server_negotiation.cc:407] Connection header received
0811 04:41:52.856651 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.856655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.856709 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.856773 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:52.861037 (+ 4264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.862103 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.863556 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.863915 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.863990 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.866355 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:52.866360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:52.866363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:52.866387 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:52.873805 (+ 7418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.874265 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.874268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.874270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.874357 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:52.874704 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:52.874707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:52.874708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:52.874883 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:52.874988 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.875223 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.875397 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1451,"thread_start_us":94,"threads_started":1}
I20260811 04:41:52.878584 29800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:52.881573 29800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:52.884307 31217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: NON_VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: true } } }
I20260811 04:41:52.884471 31217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.888237 31217 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: attempting to promote NON_VOTER 88c9f5c55e8c499cb9efa19599b946a2 to VOTER
I20260811 04:41:52.888595 31217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } }
I20260811 04:41:52.889957 30688 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 LEARNER]: Refusing update from remote peer 0c32601fed6840f1898841616a8a8815: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 04:41:52.890741 30481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Refusing update from remote peer 0c32601fed6840f1898841616a8a8815: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 04:41:52.890467 31216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 04:41:52.892227 31216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 04:41:52.892918 31226 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.893425 30688 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.894902 30481 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.895159 31226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.895287 31226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.895480 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.895633 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.895684 30732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.895787 30732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:52.896387 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.896533 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:52.896833 31227 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c32601fed6840f1898841616a8a8815" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:52.897087 31227 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:52.905457 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.898789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43109 (local address 127.0.0.1:41500)
0811 04:41:52.898978 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.899017 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.899125 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.901204 (+ 2079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.901206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.901224 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:52.901315 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.901320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.902869 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.902871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.903846 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.903854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.905026 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.905030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.905122 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.905152 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.905239 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.905317 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 04:41:52.905684 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.899109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41500 (local address 127.22.144.130:43109)
0811 04:41:52.900643 (+ 1534us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.900689 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.900709 (+ 20us) server_negotiation.cc:407] Connection header received
0811 04:41:52.900787 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.900793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.901024 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.901221 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:52.901412 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.902767 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.903974 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.904894 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.905031 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.905163 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.905295 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.905404 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1442,"thread_start_us":258,"threads_started":1}
I20260811 04:41:52.906629 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.899309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39975 (local address 127.0.0.1:41688)
0811 04:41:52.899611 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.900170 (+ 559us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.900325 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.901400 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.901402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.901411 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:52.901459 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.901462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.903954 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.903956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.904679 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.904686 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.906212 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.906216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.906341 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.906365 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.906429 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.906495 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":20,"thread_start_us":101,"threads_started":1}
I20260811 04:41:52.907420 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.899739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.22.144.131:39975)
0811 04:41:52.900643 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.900658 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.900697 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:41:52.900787 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.900793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.901030 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.901221 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:52.901639 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.902758 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.904807 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.906067 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.906889 (+ 822us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.907019 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.907091 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.907197 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":213,"threads_started":1}
I20260811 04:41:52.908594 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.899309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39011 (local address 127.0.0.1:46460)
0811 04:41:52.899998 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:52.900012 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:52.900093 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:52.903514 (+ 3421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:52.903517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:52.903537 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:52.903627 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.903632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.905815 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.905819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.907000 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:52.907011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.908268 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.908271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:52.908338 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.908357 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 04:41:52.908409 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 04:41:52.908458 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":457,"thread_start_us":97,"threads_started":1}
I20260811 04:41:52.908677 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:52.899565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.22.144.129:39011)
0811 04:41:52.903048 (+ 3483us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:52.903104 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:52.903125 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:41:52.903190 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:52.903194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:52.903347 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:52.903499 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:52.903728 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.904882 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.907137 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:52.908139 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:52.908269 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:52.908391 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:52.908444 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 04:41:52.908540 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3376,"thread_start_us":71,"threads_started":1}
I20260811 04:41:52.909320 30962 tablet_service.cc:1467] Processing CreateTablet for tablet 650d6b46d77343a78a546d87dfab56d6 (DEFAULT_TABLE table=test-table [id=7f1fce51cca54f8ab790347f64e2b4b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:52.910776 30962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 650d6b46d77343a78a546d87dfab56d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:52.911707 31123 tablet_service.cc:1467] Processing CreateTablet for tablet 650d6b46d77343a78a546d87dfab56d6 (DEFAULT_TABLE table=test-table [id=7f1fce51cca54f8ab790347f64e2b4b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:52.911794 30801 tablet_service.cc:1467] Processing CreateTablet for tablet 650d6b46d77343a78a546d87dfab56d6 (DEFAULT_TABLE table=test-table [id=7f1fce51cca54f8ab790347f64e2b4b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:52.912653 31123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 650d6b46d77343a78a546d87dfab56d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:52.912771 30801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 650d6b46d77343a78a546d87dfab56d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:52.919752 31234 tablet_bootstrap.cc:492] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Bootstrap starting.
I20260811 04:41:52.921355 31237 tablet_bootstrap.cc:492] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Bootstrap starting.
I20260811 04:41:52.922204 31235 tablet_bootstrap.cc:492] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Bootstrap starting.
I20260811 04:41:52.923087 31234 tablet_bootstrap.cc:654] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:52.923738 31237 tablet_bootstrap.cc:654] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:52.923946 31234 log.cc:824] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:52.924441 31235 tablet_bootstrap.cc:654] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:52.924930 31237 log.cc:824] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:52.925689 31235 log.cc:824] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:52.926235 31234 tablet_bootstrap.cc:492] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: No bootstrap required, opened a new log
I20260811 04:41:52.926553 31234 ts_tablet_manager.cc:1397] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 04:41:52.927156 31237 tablet_bootstrap.cc:492] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: No bootstrap required, opened a new log
I20260811 04:41:52.927354 31237 ts_tablet_manager.cc:1397] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:41:52.928081 31235 tablet_bootstrap.cc:492] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: No bootstrap required, opened a new log
I20260811 04:41:52.928328 31235 ts_tablet_manager.cc:1397] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 04:41:52.930882 31234 raft_consensus.cc:348] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:52.931190 31234 raft_consensus.cc:374] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:52.931248 31234 raft_consensus.cc:729] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2542949f0b5440c9418f4fd18486d2d, State: Initialized, Role: FOLLOWER
I20260811 04:41:52.931721 31234 consensus_queue.cc:260] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:52.932293 31237 raft_consensus.cc:348] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:52.932639 31199 heartbeater.cc:503] Master 127.22.144.190:42529 was elected leader, sending a full tablet report...
I20260811 04:41:52.932673 31237 raft_consensus.cc:374] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:52.932731 31237 raft_consensus.cc:729] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf3a8a7328e4e5a84322e7b0c1a713a, State: Initialized, Role: FOLLOWER
I20260811 04:41:52.933178 31237 consensus_queue.cc:260] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:52.934213 31237 ts_tablet_manager.cc:1428] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 04:41:52.934401 30880 heartbeater.cc:503] Master 127.22.144.190:42529 was elected leader, sending a full tablet report...
I20260811 04:41:52.938205 31234 ts_tablet_manager.cc:1428] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260811 04:41:52.939131 31235 raft_consensus.cc:348] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:52.939355 31235 raft_consensus.cc:374] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:52.939426 31235 raft_consensus.cc:729] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99979722e0764015a2801f38a06f7e31, State: Initialized, Role: FOLLOWER
I20260811 04:41:52.939934 31235 consensus_queue.cc:260] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:52.940687 31042 heartbeater.cc:503] Master 127.22.144.190:42529 was elected leader, sending a full tablet report...
I20260811 04:41:52.941962 31235 ts_tablet_manager.cc:1428] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Time spent starting tablet: real 0.014s user 0.000s sys 0.007s
I20260811 04:41:53.058471 31242 raft_consensus.cc:482] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:53.058820 31242 raft_consensus.cc:504] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:53.060052 31242 leader_election.cc:288] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109), ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011)
I20260811 04:41:53.065407 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.060506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:36241 (local address 127.22.144.129:39011)
0811 04:41:53.060629 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.060634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.060774 (+ 140us) server_negotiation.cc:407] Connection header received
0811 04:41:53.060849 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.060852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.060922 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.061030 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:53.061405 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.062190 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.063993 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.064630 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.064810 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.064869 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.065187 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.065273 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:41:53.065407 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.060593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:53913 (local address 127.22.144.130:43109)
0811 04:41:53.060714 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.060720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.060867 (+ 147us) server_negotiation.cc:407] Connection header received
0811 04:41:53.060931 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.060934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.060986 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.061079 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:53.061394 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.062183 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.063993 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.064722 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.064830 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.064875 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.065187 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.065281 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 04:41:53.065411 31203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.060381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39011 (local address 127.22.144.131:36241)
0811 04:41:53.060541 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:53.060629 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:53.060755 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:53.061039 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:53.061041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:53.061057 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:53.061257 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:53.061264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.062311 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.062317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:53.063830 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:53.063841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.064831 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.064835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:53.065007 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.065048 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 04:41:53.065131 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 04:41:53.065246 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":142}
I20260811 04:41:53.065412 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.060380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43109 (local address 127.22.144.131:53913)
0811 04:41:53.060712 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:53.060735 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:53.060838 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:53.061108 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:53.061111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:53.061128 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:53.061262 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:53.061277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.062295 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.062300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:53.063830 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:53.063841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.064842 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.064845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:53.065002 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.065053 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 04:41:53.065131 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 04:41:53.065246 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":387}
I20260811 04:41:53.066287 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d6b46d77343a78a546d87dfab56d6" candidate_uuid: "f2542949f0b5440c9418f4fd18486d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99979722e0764015a2801f38a06f7e31" is_pre_election: true
I20260811 04:41:53.066498 30821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d6b46d77343a78a546d87dfab56d6" candidate_uuid: "f2542949f0b5440c9418f4fd18486d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" is_pre_election: true
I20260811 04:41:53.066623 30987 raft_consensus.cc:2454] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2542949f0b5440c9418f4fd18486d2d in term 0.
I20260811 04:41:53.066828 30821 raft_consensus.cc:2454] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2542949f0b5440c9418f4fd18486d2d in term 0.
I20260811 04:41:53.067242 31071 leader_election.cc:302] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99979722e0764015a2801f38a06f7e31, f2542949f0b5440c9418f4fd18486d2d; no voters:
I20260811 04:41:53.067634 31242 raft_consensus.cc:2781] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:53.067716 31242 raft_consensus.cc:482] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:53.067746 31242 raft_consensus.cc:3037] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:53.069028 31242 raft_consensus.cc:504] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:53.069427 31242 leader_election.cc:288] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [CANDIDATE]: Term 1 election: Requested vote from peers 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109), ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011)
I20260811 04:41:53.069903 30821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d6b46d77343a78a546d87dfab56d6" candidate_uuid: "f2542949f0b5440c9418f4fd18486d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a"
I20260811 04:41:53.069999 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d6b46d77343a78a546d87dfab56d6" candidate_uuid: "f2542949f0b5440c9418f4fd18486d2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99979722e0764015a2801f38a06f7e31"
I20260811 04:41:53.070067 30821 raft_consensus.cc:3037] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:53.070145 30987 raft_consensus.cc:3037] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:53.071326 30821 raft_consensus.cc:2454] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2542949f0b5440c9418f4fd18486d2d in term 1.
I20260811 04:41:53.071419 30987 raft_consensus.cc:2454] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2542949f0b5440c9418f4fd18486d2d in term 1.
I20260811 04:41:53.071720 31068 leader_election.cc:302] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecf3a8a7328e4e5a84322e7b0c1a713a, f2542949f0b5440c9418f4fd18486d2d; no voters:
I20260811 04:41:53.071897 31242 raft_consensus.cc:2781] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:53.072155 31242 raft_consensus.cc:686] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 1 LEADER]: Becoming Leader. State: Replica: f2542949f0b5440c9418f4fd18486d2d, State: Running, Role: LEADER
I20260811 04:41:53.072521 31242 consensus_queue.cc:237] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } }
I20260811 04:41:53.076084 29801 catalog_manager.cc:5495] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d reported cstate change: term changed from 0 to 1, leader changed from <none> to f2542949f0b5440c9418f4fd18486d2d (127.22.144.131). New cstate: current_term: 1 leader_uuid: "f2542949f0b5440c9418f4fd18486d2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } health_report { overall_health: HEALTHY } } }
W20260811 04:41:53.093421 31201 tablet.cc:2369] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:53.095741 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.092027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.22.144.131:39975)
0811 04:41:53.092157 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.092161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.092174 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:53.092244 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.092246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.092298 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.092386 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:53.092677 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.093170 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.094194 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.094443 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.094599 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.094849 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:53.095255 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:53.095349 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.095450 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.095504 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:41:53.102005 30821 raft_consensus.cc:1261] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Refusing update from remote peer f2542949f0b5440c9418f4fd18486d2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:53.102005 30987 raft_consensus.cc:1261] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Refusing update from remote peer f2542949f0b5440c9418f4fd18486d2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 04:41:53.102696 31044 tablet.cc:2369] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:53.102770 31245 consensus_queue.cc:1035] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:53.103053 31242 consensus_queue.cc:1035] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:53.113121 31251 mvcc.cc:205] Tried to move back new op lower bound from 7317189890458464256 to 7317189890348056576. Current Snapshot: MvccSnapshot[applied={T|T < 7317189890458464256}]
I20260811 04:41:53.125120 29800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:53.125383 29800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:53.131036 29800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c32601fed6840f1898841616a8a8815: Sending DeleteTablet for 3 replicas of tablet 650d6b46d77343a78a546d87dfab56d6
I20260811 04:41:53.131829 30962 tablet_service.cc:1514] Processing DeleteTablet for tablet 650d6b46d77343a78a546d87dfab56d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:53 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:41500
I20260811 04:41:53.132416 31263 tablet_replica.cc:330] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: stopping tablet replica
I20260811 04:41:53.132797 31263 raft_consensus.cc:2229] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:53.133263 31263 raft_consensus.cc:2258] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:53.136310 30801 tablet_service.cc:1514] Processing DeleteTablet for tablet 650d6b46d77343a78a546d87dfab56d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:53 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:46460
I20260811 04:41:53.137015 31265 tablet_replica.cc:330] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: stopping tablet replica
I20260811 04:41:53.137440 31265 raft_consensus.cc:2229] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:53.137440 31123 tablet_service.cc:1514] Processing DeleteTablet for tablet 650d6b46d77343a78a546d87dfab56d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:53 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:41688
I20260811 04:41:53.137926 31265 raft_consensus.cc:2258] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:53.138134 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 29770
I20260811 04:41:53.142185 31265 ts_tablet_manager.cc:1905] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:53.145612 31265 ts_tablet_manager.cc:1918] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:53.145819 31265 log.cc:1196] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/650d6b46d77343a78a546d87dfab56d6
I20260811 04:41:53.147344 31263 ts_tablet_manager.cc:1905] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:53.150120 31263 ts_tablet_manager.cc:1918] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:53.150223 31263 log.cc:1196] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/650d6b46d77343a78a546d87dfab56d6
I20260811 04:41:53.150241 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.133001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.22.144.189:37873)
0811 04:41:53.133160 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.133166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.135124 (+ 1958us) server_negotiation.cc:407] Connection header received
0811 04:41:53.135358 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.135361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.135413 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.135471 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:53.137115 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.137965 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.142660 (+ 4695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.143065 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.143179 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.145656 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:53.145663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:53.145669 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:53.145709 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:53.147784 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.148431 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.148438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.148443 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.148544 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.148911 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.148917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.148922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.149200 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:53.149327 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.149788 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.149939 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 04:41:53.150537 31263 ts_tablet_manager.cc:1939] T 650d6b46d77343a78a546d87dfab56d6 P 99979722e0764015a2801f38a06f7e31: Deleting consensus metadata
I20260811 04:41:53.152956 31266 tablet_replica.cc:330] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: stopping tablet replica
I20260811 04:41:53.153329 31266 raft_consensus.cc:2229] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 1 LEADER]: Raft consensus shutting down.
W20260811 04:41:53.153646 30908 connection.cc:441] server connection from 127.0.0.1:41500 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41500 (request call id 1) could send its response
I20260811 04:41:53.153914 31266 raft_consensus.cc:2258] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:53.154243 31265 ts_tablet_manager.cc:1939] T 650d6b46d77343a78a546d87dfab56d6 P ecf3a8a7328e4e5a84322e7b0c1a713a: Deleting consensus metadata
W20260811 04:41:53.155478 30749 connection.cc:441] server connection from 127.0.0.1:46460 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46460 (request call id 1) could send its response
I20260811 04:41:53.161067 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.135231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.22.144.188:40639)
0811 04:41:53.135365 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.135369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.150994 (+ 15625us) server_negotiation.cc:407] Connection header received
0811 04:41:53.151214 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.151220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.151287 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.151344 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:53.152353 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.153116 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.154583 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.154913 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.155000 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.156851 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:53.156856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:53.156859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:53.156890 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:53.158744 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.159495 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.159499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.159501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.159584 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.159891 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.159894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.159896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.160118 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:53.160243 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.160776 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.160895 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 04:41:53.162948 31266 ts_tablet_manager.cc:1905] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:53.166169 31266 ts_tablet_manager.cc:1918] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:53.166286 31266 log.cc:1196] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/650d6b46d77343a78a546d87dfab56d6
I20260811 04:41:53.166706 31266 ts_tablet_manager.cc:1939] T 650d6b46d77343a78a546d87dfab56d6 P f2542949f0b5440c9418f4fd18486d2d: Deleting consensus metadata
W20260811 04:41:53.167937 31068 connection.cc:441] server connection from 127.0.0.1:41688 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41688 (request call id 1) could send its response
I20260811 04:41:53.169158 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.159318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.22.144.188:40639)
0811 04:41:53.159442 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.159446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.159459 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:53.159645 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.159651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.159708 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.159763 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:53.161036 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.161855 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.163045 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.163354 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.163473 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.165333 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:53.165337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:53.165340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:53.165365 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:53.167211 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.167750 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.167754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.167755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.167844 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.168227 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.168231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.168233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.168419 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:53.168574 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.168851 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.168970 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 04:41:53.169847 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:53.158215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.22.144.189:37873)
0811 04:41:53.158380 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:53.158385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:53.159131 (+ 746us) server_negotiation.cc:407] Connection header received
0811 04:41:53.159327 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:53.159333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:53.159386 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:53.159437 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:53.161460 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.162273 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.163676 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:53.164019 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:53.164138 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:53.165942 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:53.165946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:53.165949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:53.165975 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:53.167939 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.168433 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.168437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.168439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.168521 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:53.168872 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:53.168876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:53.168878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:53.169093 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:53.169187 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:53.169578 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 04:41:53.169695 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 04:41:54.116734 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.116178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:39793)
0811 04:41:54.116498 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.116613 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":117,"threads_started":1}
W20260811 04:41:54.117635 31199 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:42529 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
I20260811 04:41:54.118566 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.117922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:56177)
0811 04:41:54.118353 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.118451 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":71,"threads_started":1}
I20260811 04:41:54.118628 31275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.117768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:35537)
0811 04:41:54.118387 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.118498 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":74,"threads_started":1}
W20260811 04:41:54.119184 30880 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:42529 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
W20260811 04:41:54.119216 31042 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:42529 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
I20260811 04:41:54.631599 31277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0c32601fed6840f1898841616a8a8815)
I20260811 04:41:54.631892 31277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } }
I20260811 04:41:54.633287 31277 leader_election.cc:288] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529), c3e01731b6ca4c6caf87db0ce8453d4e (127.22.144.189:37873)
I20260811 04:41:54.633275 31278 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0c32601fed6840f1898841616a8a8815)
I20260811 04:41:54.633543 31278 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } }
I20260811 04:41:54.634124 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.633463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59928)
0811 04:41:54.633834 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.633972 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":85,"threads_started":1}
W20260811 04:41:54.634311 30658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
I20260811 04:41:54.635257 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.634941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59952)
0811 04:41:54.635072 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.635183 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 04:41:54.635396 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.634704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59938)
0811 04:41:54.635222 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.635296 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":69,"threads_started":1}
W20260811 04:41:54.635371 30658 leader_election.cc:334] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529): Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
W20260811 04:41:54.635568 30453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
I20260811 04:41:54.635725 31278 leader_election.cc:288] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529), 88c9f5c55e8c499cb9efa19599b946a2 (127.22.144.188:40639)
I20260811 04:41:54.636619 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.636347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59960)
0811 04:41:54.636483 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.636549 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 04:41:54.636807 30453 leader_election.cc:334] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529): Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
I20260811 04:41:54.638785 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.633544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:37873 (local address 127.0.0.1:46082)
0811 04:41:54.634207 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.634234 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.634332 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.634816 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.634822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.634850 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.634985 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.634993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.636526 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.636530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.637601 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.637612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.638345 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.638348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.638440 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.638467 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.638550 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.638643 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":73,"threads_started":1}
I20260811 04:41:54.638784 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.633986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.22.144.189:37873)
0811 04:41:54.634548 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.634554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.634569 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:54.634617 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.634634 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.634687 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.634774 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.635145 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.636407 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.637770 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.638218 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.638322 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.638363 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.638577 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.638663 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":74,"threads_started":1}
I20260811 04:41:54.639374 30481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "88c9f5c55e8c499cb9efa19599b946a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" is_pre_election: true
I20260811 04:41:54.639621 30481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c9f5c55e8c499cb9efa19599b946a2 in term 1.
I20260811 04:41:54.639952 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.635546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.22.144.188:40639)
0811 04:41:54.635960 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.635972 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.635986 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:41:54.636066 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.636069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.636115 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.636216 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.636787 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.637420 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.638700 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.639372 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.639517 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.639565 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.639746 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.639824 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":114,"threads_started":1}
I20260811 04:41:54.639983 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.635218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:40639 (local address 127.0.0.1:40908)
0811 04:41:54.635680 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.635840 (+ 160us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.635933 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.636479 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.636485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.636509 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.636599 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.636605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.637601 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.637604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.638577 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.638587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.639489 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.639492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.639611 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.639639 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.639702 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.639795 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":51,"threads_started":1}
I20260811 04:41:54.640033 30655 leader_election.cc:302] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88c9f5c55e8c499cb9efa19599b946a2, c3e01731b6ca4c6caf87db0ce8453d4e; no voters: 0c32601fed6840f1898841616a8a8815
I20260811 04:41:54.640415 30688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88c9f5c55e8c499cb9efa19599b946a2" is_pre_election: true
I20260811 04:41:54.640424 31277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 04:41:54.640599 31277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0c32601fed6840f1898841616a8a8815)
I20260811 04:41:54.640658 31277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 04:41:54.641882 31277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } }
I20260811 04:41:54.642184 30688 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3e01731b6ca4c6caf87db0ce8453d4e in current term 2: Already voted for candidate 88c9f5c55e8c499cb9efa19599b946a2 in this term.
I20260811 04:41:54.642340 31277 leader_election.cc:288] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [CANDIDATE]: Term 2 election: Requested vote from peers 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529), c3e01731b6ca4c6caf87db0ce8453d4e (127.22.144.189:37873)
I20260811 04:41:54.642694 30481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "88c9f5c55e8c499cb9efa19599b946a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e"
I20260811 04:41:54.642827 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.642472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59976)
0811 04:41:54.642669 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.642754 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260811 04:41:54.642853 30481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 1 FOLLOWER]: Advancing to term 2
I20260811 04:41:54.643044 30455 leader_election.cc:302] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3e01731b6ca4c6caf87db0ce8453d4e; no voters: 0c32601fed6840f1898841616a8a8815, 88c9f5c55e8c499cb9efa19599b946a2
I20260811 04:41:54.643599 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.643346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59980)
0811 04:41:54.643472 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.643538 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 04:41:54.643703 30658 leader_election.cc:334] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529): Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
I20260811 04:41:54.644112 30481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c9f5c55e8c499cb9efa19599b946a2 in term 2.
I20260811 04:41:54.644215 31278 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:41:54.644472 30655 leader_election.cc:302] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88c9f5c55e8c499cb9efa19599b946a2, c3e01731b6ca4c6caf87db0ce8453d4e; no voters: 0c32601fed6840f1898841616a8a8815
I20260811 04:41:54.644668 31277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 04:41:54.644914 31277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [term 2 LEADER]: Becoming Leader. State: Replica: 88c9f5c55e8c499cb9efa19599b946a2, State: Running, Role: LEADER
I20260811 04:41:54.645207 31277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } }
I20260811 04:41:54.646682 31287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88c9f5c55e8c499cb9efa19599b946a2. Latest consensus state: current_term: 2 leader_uuid: "88c9f5c55e8c499cb9efa19599b946a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:54.646801 31287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:54.647116 31289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:54.649464 31289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:54.649887 31289 catalog_manager.cc:1259] Loaded cluster ID: 9e8412f884224e7b940d99127e9d9971
I20260811 04:41:54.649935 31289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:54.650440 31289 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:54.650807 31289 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Loaded TSK: 0
I20260811 04:41:54.651446 31289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:41:54.693769 30677 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:54.696898 30677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:54.700258 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.699946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:59994)
0811 04:41:54.700108 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.700176 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 04:41:54.700304 30481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [term 2 FOLLOWER]: Refusing update from remote peer 88c9f5c55e8c499cb9efa19599b946a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 04:41:54.700848 31287 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 04:41:54.701146 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.700883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.0.0.1:60000)
0811 04:41:54.700987 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.701064 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 04:41:54.701278 30658 consensus_peers.cc:487] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 -> Peer 0c32601fed6840f1898841616a8a8815 (127.22.144.190:42529): Couldn't send request to peer 0c32601fed6840f1898841616a8a8815. Status: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 04:41:54.702607 31278 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88c9f5c55e8c499cb9efa19599b946a2. Latest consensus state: current_term: 2 leader_uuid: "88c9f5c55e8c499cb9efa19599b946a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:54.702765 31278 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:54.703778 31290 mvcc.cc:205] Tried to move back new op lower bound from 7317189897006809088 to 7317189896789659648. Current Snapshot: MvccSnapshot[applied={T|T < 7317189897006809088}]
I20260811 04:41:54.703936 31277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "88c9f5c55e8c499cb9efa19599b946a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:54.704075 31277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:54.704416 31291 mvcc.cc:205] Tried to move back new op lower bound from 7317189897006809088 to 7317189896789659648. Current Snapshot: MvccSnapshot[applied={T|T < 7317189897006809088}]
I20260811 04:41:54.704599 31278 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "88c9f5c55e8c499cb9efa19599b946a2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0c32601fed6840f1898841616a8a8815" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42529 } } peers { permanent_uuid: "c3e01731b6ca4c6caf87db0ce8453d4e" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 37873 } attrs { promote: false } } peers { permanent_uuid: "88c9f5c55e8c499cb9efa19599b946a2" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 40639 } attrs { promote: false } } }
I20260811 04:41:54.704733 31278 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e01731b6ca4c6caf87db0ce8453d4e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:41:54.714823 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.710380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39011 (local address 127.0.0.1:46474)
0811 04:41:54.710479 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.710498 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.710644 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.711263 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.711266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.711285 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.711422 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.711429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.712571 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.712577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.713364 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.713373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.714422 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.714426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.714513 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.714544 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.714606 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.714679 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1}
I20260811 04:41:54.715394 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.710969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43109 (local address 127.0.0.1:41516)
0811 04:41:54.711112 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.711125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.711212 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.711619 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.711622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.711642 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.711752 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.711758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.712892 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.712895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.714101 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.714112 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.715007 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.715012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.715134 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.715158 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.715210 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.715275 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":26}
I20260811 04:41:54.715425 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.710642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.22.144.129:39011)
0811 04:41:54.710972 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.710977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.710992 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:54.711056 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.711060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.711121 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.711570 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.711708 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.712486 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.713520 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.714297 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.715132 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.715188 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.715226 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.715296 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":103,"threads_started":1}
I20260811 04:41:54.715682 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.711028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39975 (local address 127.0.0.1:41700)
0811 04:41:54.711930 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.711945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.712040 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.712288 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.712291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.712307 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.712403 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.712408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.713359 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.713363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.714350 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.714359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.715269 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.715272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.715347 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.715365 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.715460 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.715553 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":22,"thread_start_us":93,"threads_started":1}
I20260811 04:41:54.715914 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.711094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41516 (local address 127.22.144.130:43109)
0811 04:41:54.711355 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.711360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.711376 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:54.711426 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.711429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.711478 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.711863 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.711968 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.712762 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.714245 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.714863 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.715612 (+ 749us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.715673 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.715709 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.715779 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260811 04:41:54.716023 30801 tablet_service.cc:1467] Processing CreateTablet for tablet 60f5db9450b44aebb19982d2c3f89057 (DEFAULT_TABLE table=test-table [id=d6be8d5b307b4ff9b6c007618b19d724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:54.716225 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.711405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.22.144.131:39975)
0811 04:41:54.711685 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.711690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.712036 (+ 346us) server_negotiation.cc:407] Connection header received
0811 04:41:54.712124 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.712126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.712173 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.712264 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.712523 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.713233 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.714494 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.715046 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.715158 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.715216 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.715980 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.716080 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260811 04:41:54.716337 30801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f5db9450b44aebb19982d2c3f89057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:54.716395 30962 tablet_service.cc:1467] Processing CreateTablet for tablet 60f5db9450b44aebb19982d2c3f89057 (DEFAULT_TABLE table=test-table [id=d6be8d5b307b4ff9b6c007618b19d724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:54.716651 30962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f5db9450b44aebb19982d2c3f89057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:54.716698 31123 tablet_service.cc:1467] Processing CreateTablet for tablet 60f5db9450b44aebb19982d2c3f89057 (DEFAULT_TABLE table=test-table [id=d6be8d5b307b4ff9b6c007618b19d724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:54.716984 31123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f5db9450b44aebb19982d2c3f89057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:54.719141 31299 tablet_bootstrap.cc:492] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a: Bootstrap starting.
I20260811 04:41:54.719748 31300 tablet_bootstrap.cc:492] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Bootstrap starting.
I20260811 04:41:54.719715 31301 tablet_bootstrap.cc:492] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Bootstrap starting.
I20260811 04:41:54.720274 31299 tablet_bootstrap.cc:654] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:54.720645 31300 tablet_bootstrap.cc:654] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:54.721007 31301 tablet_bootstrap.cc:654] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:54.721668 31299 tablet_bootstrap.cc:492] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a: No bootstrap required, opened a new log
I20260811 04:41:54.721786 31299 ts_tablet_manager.cc:1397] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:54.721923 31300 tablet_bootstrap.cc:492] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: No bootstrap required, opened a new log
I20260811 04:41:54.722040 31300 ts_tablet_manager.cc:1397] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 04:41:54.722237 31301 tablet_bootstrap.cc:492] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: No bootstrap required, opened a new log
I20260811 04:41:54.722332 31301 ts_tablet_manager.cc:1397] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 04:41:54.722307 31299 raft_consensus.cc:348] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.722430 31299 raft_consensus.cc:374] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:54.722472 31299 raft_consensus.cc:729] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf3a8a7328e4e5a84322e7b0c1a713a, State: Initialized, Role: FOLLOWER
I20260811 04:41:54.722514 31300 raft_consensus.cc:348] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.722641 31300 raft_consensus.cc:374] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:54.722606 31299 consensus_queue.cc:260] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.722688 31300 raft_consensus.cc:729] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99979722e0764015a2801f38a06f7e31, State: Initialized, Role: FOLLOWER
I20260811 04:41:54.722754 31301 raft_consensus.cc:348] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.722870 31301 raft_consensus.cc:374] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:54.722828 31300 consensus_queue.cc:260] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.722914 31301 raft_consensus.cc:729] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2542949f0b5440c9418f4fd18486d2d, State: Initialized, Role: FOLLOWER
I20260811 04:41:54.722983 31299 ts_tablet_manager.cc:1428] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:54.723191 31300 ts_tablet_manager.cc:1428] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 04:41:54.723834 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.723408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:48693)
0811 04:41:54.723647 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.723713 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":101,"threads_started":1}
I20260811 04:41:54.725036 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.723899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:44253)
0811 04:41:54.724873 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.724937 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":94,"threads_started":1}
I20260811 04:41:54.725951 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.725697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:47631)
0811 04:41:54.725827 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.725891 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 04:41:54.726755 30879 heartbeater.cc:503] Master 127.22.144.188:40639 was elected leader, sending a full tablet report...
I20260811 04:41:54.726992 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.724368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:58449)
0811 04:41:54.726877 (+ 2509us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.726935 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2411}
I20260811 04:41:54.727154 31041 heartbeater.cc:503] Master 127.22.144.188:40639 was elected leader, sending a full tablet report...
I20260811 04:41:54.723073 31301 consensus_queue.cc:260] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.728116 31301 ts_tablet_manager.cc:1428] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 04:41:54.728745 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.728336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:49725)
0811 04:41:54.728591 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.728651 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
I20260811 04:41:54.729645 31197 heartbeater.cc:503] Master 127.22.144.188:40639 was elected leader, sending a full tablet report...
I20260811 04:41:54.729908 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.729650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:56785)
0811 04:41:54.729795 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.729843 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 04:41:54.844851 31306 raft_consensus.cc:482] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:54.845041 31306 raft_consensus.cc:504] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.846458 31306 leader_election.cc:288] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975), 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109)
I20260811 04:41:54.850528 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.846683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39975 (local address 127.22.144.129:49649)
0811 04:41:54.846815 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.846959 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.847112 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.847407 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.847410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.847428 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.847560 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.847565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.848479 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.848485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.849358 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.849368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.850081 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.850085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.850198 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.850227 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.850321 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.850394 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1}
I20260811 04:41:54.850528 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.846793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:49649 (local address 127.22.144.131:39975)
0811 04:41:54.846920 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.846927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.847032 (+ 105us) server_negotiation.cc:407] Connection header received
0811 04:41:54.847255 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.847258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.847306 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.847393 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.847771 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.848366 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.849520 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.849924 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.850070 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.850142 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.850337 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.850407 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 04:41:54.850868 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.846794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43109 (local address 127.22.144.129:42477)
0811 04:41:54.847153 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.847169 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:54.847291 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:54.847599 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:54.847603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:54.847621 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:54.847768 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.847774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.848667 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.848672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.849656 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:54.849666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.850371 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.850375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:54.850480 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.850504 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:41:54.850619 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 04:41:54.850720 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":24,"thread_start_us":161,"threads_started":1}
I20260811 04:41:54.851094 31145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f5db9450b44aebb19982d2c3f89057" candidate_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2542949f0b5440c9418f4fd18486d2d" is_pre_election: true
I20260811 04:41:54.851188 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.847016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:42477 (local address 127.22.144.130:43109)
0811 04:41:54.847159 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.847179 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.847239 (+ 60us) server_negotiation.cc:407] Connection header received
0811 04:41:54.847429 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.847433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.847489 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.847585 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:54.847920 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.848541 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.849787 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.850233 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.850377 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.850431 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.850937 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.851039 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 04:41:54.851305 31145 raft_consensus.cc:2454] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecf3a8a7328e4e5a84322e7b0c1a713a in term 0.
I20260811 04:41:54.851707 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f5db9450b44aebb19982d2c3f89057" candidate_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99979722e0764015a2801f38a06f7e31" is_pre_election: true
I20260811 04:41:54.851706 30749 leader_election.cc:302] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecf3a8a7328e4e5a84322e7b0c1a713a, f2542949f0b5440c9418f4fd18486d2d; no voters:
I20260811 04:41:54.851876 30987 raft_consensus.cc:2454] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecf3a8a7328e4e5a84322e7b0c1a713a in term 0.
I20260811 04:41:54.852026 31306 raft_consensus.cc:2781] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:54.852097 31306 raft_consensus.cc:482] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:54.852144 31306 raft_consensus.cc:3037] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:54.853365 31306 raft_consensus.cc:504] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.853826 31306 leader_election.cc:288] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [CANDIDATE]: Term 1 election: Requested vote from peers f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975), 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109)
I20260811 04:41:54.854190 30987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f5db9450b44aebb19982d2c3f89057" candidate_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99979722e0764015a2801f38a06f7e31"
I20260811 04:41:54.854190 31145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f5db9450b44aebb19982d2c3f89057" candidate_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2542949f0b5440c9418f4fd18486d2d"
I20260811 04:41:54.854326 30987 raft_consensus.cc:3037] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:54.854350 31145 raft_consensus.cc:3037] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:54.855579 31145 raft_consensus.cc:2454] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecf3a8a7328e4e5a84322e7b0c1a713a in term 1.
I20260811 04:41:54.855574 30987 raft_consensus.cc:2454] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecf3a8a7328e4e5a84322e7b0c1a713a in term 1.
I20260811 04:41:54.855886 30749 leader_election.cc:302] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ecf3a8a7328e4e5a84322e7b0c1a713a, f2542949f0b5440c9418f4fd18486d2d; no voters:
I20260811 04:41:54.856101 31306 raft_consensus.cc:2781] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:54.856390 31306 raft_consensus.cc:686] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 LEADER]: Becoming Leader. State: Replica: ecf3a8a7328e4e5a84322e7b0c1a713a, State: Running, Role: LEADER
I20260811 04:41:54.856678 31306 consensus_queue.cc:237] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } }
I20260811 04:41:54.857172 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.856891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:44419)
0811 04:41:54.857023 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.857097 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 04:41:54.859568 30677 catalog_manager.cc:5495] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a reported cstate change: term changed from 0 to 1, leader changed from <none> to ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129). New cstate: current_term: 1 leader_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf3a8a7328e4e5a84322e7b0c1a713a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 } health_report { overall_health: UNKNOWN } } }
I20260811 04:41:54.874029 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.870392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.22.144.129:39011)
0811 04:41:54.870529 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:54.870534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:54.870591 (+ 57us) server_negotiation.cc:407] Connection header received
0811 04:41:54.870672 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:54.870675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:54.870732 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:54.870831 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:54.871083 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.871700 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.872569 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:54.872807 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:54.872898 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:54.873157 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:54.873707 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:54.873804 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:54.873870 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 04:41:54.873920 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 04:41:54.877147 30987 raft_consensus.cc:1261] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Refusing update from remote peer ecf3a8a7328e4e5a84322e7b0c1a713a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:54.877398 31145 raft_consensus.cc:1261] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Refusing update from remote peer ecf3a8a7328e4e5a84322e7b0c1a713a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:54.877771 31313 consensus_queue.cc:1035] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [LEADER]: Connected to new peer: Peer: permanent_uuid: "99979722e0764015a2801f38a06f7e31" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:54.878067 31306 consensus_queue.cc:1035] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2542949f0b5440c9418f4fd18486d2d" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:54.879887 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.879428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:45909)
0811 04:41:54.879737 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.879802 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260811 04:41:54.881029 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.880686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:35449)
0811 04:41:54.880898 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.880960 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 04:41:54.883978 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.883690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.130:46977)
0811 04:41:54.883840 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.883897 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 04:41:54.884234 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.883978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.129:56119)
0811 04:41:54.884100 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.884160 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 04:41:54.884718 31320 mvcc.cc:205] Tried to move back new op lower bound from 7317189897731223552 to 7317189897656328192. Current Snapshot: MvccSnapshot[applied={T|T < 7317189897731223552}]
I20260811 04:41:54.885720 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.885408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:42529 (local address 127.22.144.131:47659)
0811 04:41:54.885594 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.885652 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:42529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 04:41:54.893263 30676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:54.893457 30676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:54.897588 30676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88c9f5c55e8c499cb9efa19599b946a2: Sending DeleteTablet for 3 replicas of tablet 60f5db9450b44aebb19982d2c3f89057
I20260811 04:41:54.898345 30801 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f5db9450b44aebb19982d2c3f89057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:54 UTC) from {username='oryx', principal='oryx/127.22.144.188@KRBTEST.COM'} at 127.0.0.1:46474
I20260811 04:41:54.898569 31123 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f5db9450b44aebb19982d2c3f89057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:54 UTC) from {username='oryx', principal='oryx/127.22.144.188@KRBTEST.COM'} at 127.0.0.1:41700
I20260811 04:41:54.898967 30962 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f5db9450b44aebb19982d2c3f89057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:54 UTC) from {username='oryx', principal='oryx/127.22.144.188@KRBTEST.COM'} at 127.0.0.1:41516
I20260811 04:41:54.899242 31329 tablet_replica.cc:330] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a: stopping tablet replica
I20260811 04:41:54.899387 31330 tablet_replica.cc:330] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: stopping tablet replica
I20260811 04:41:54.899392 31329 raft_consensus.cc:2229] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:54.899523 31330 raft_consensus.cc:2229] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:54.899680 31330 raft_consensus.cc:2258] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:54.899763 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 30629
I20260811 04:41:54.899747 31329 raft_consensus.cc:2258] T 60f5db9450b44aebb19982d2c3f89057 P ecf3a8a7328e4e5a84322e7b0c1a713a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:54.900653 31330 ts_tablet_manager.cc:1905] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:54.903630 31330 ts_tablet_manager.cc:1918] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:54.903745 31330 log.cc:1196] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/60f5db9450b44aebb19982d2c3f89057
I20260811 04:41:54.903978 31331 tablet_replica.cc:330] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: stopping tablet replica
I20260811 04:41:54.904121 31330 ts_tablet_manager.cc:1939] T 60f5db9450b44aebb19982d2c3f89057 P f2542949f0b5440c9418f4fd18486d2d: Deleting consensus metadata
I20260811 04:41:54.904155 31331 raft_consensus.cc:2229] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:54.904325 31331 raft_consensus.cc:2258] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:54.904984 30660 catalog_manager.cc:4841] TS f2542949f0b5440c9418f4fd18486d2d (127.22.144.131:39975): tablet 60f5db9450b44aebb19982d2c3f89057 (table test-table [id=d6be8d5b307b4ff9b6c007618b19d724]) successfully deleted
I20260811 04:41:54.905169 31331 ts_tablet_manager.cc:1905] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:54.907461 31331 ts_tablet_manager.cc:1918] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:54.907536 31331 log.cc:1196] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/60f5db9450b44aebb19982d2c3f89057
I20260811 04:41:54.907825 31331 ts_tablet_manager.cc:1939] T 60f5db9450b44aebb19982d2c3f89057 P 99979722e0764015a2801f38a06f7e31: Deleting consensus metadata
I20260811 04:41:54.908727 30661 catalog_manager.cc:4841] TS 99979722e0764015a2801f38a06f7e31 (127.22.144.130:43109): tablet 60f5db9450b44aebb19982d2c3f89057 (table test-table [id=d6be8d5b307b4ff9b6c007618b19d724]) successfully deleted
I20260811 04:41:54.909639 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 30888
I20260811 04:41:54.910051 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:54.909731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39011 (local address 127.0.0.1:46490)
0811 04:41:54.909877 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:54.909955 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 04:41:54.913894 30655 catalog_manager.cc:4584] TS ecf3a8a7328e4e5a84322e7b0c1a713a (127.22.144.129:39011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60f5db9450b44aebb19982d2c3f89057: Network error: Client connection negotiation failed: client connection to 127.22.144.129:39011: connect: Connection refused (error 111)
I20260811 04:41:54.920352 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 31049
I20260811 04:41:54.929773 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 30319
I20260811 04:41:54.939761 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 30431
2026-08-11T04:41:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5772 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:55 dist-test-slave-ss4n krb5kdc[31334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:55 dist-test-slave-ss4n krb5kdc[31334](info): set up 2 sockets
Aug 11 04:41:55 dist-test-slave-ss4n krb5kdc[31334](info): commencing operation
krb5kdc: starting...
W20260811 04:41:56.114734 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:56 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab.
Entry for principal oryx/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab.
I20260811 04:41:56.265154 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:38589
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab
--principal=oryx/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:38589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:56.389000 31350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:56.389310 31350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:56.389353 31350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:56.393612 31350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:56.393684 31350 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 04:41:56.393702 31350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:56.393720 31350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:56.393738 31350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:56.393754 31350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:56.399219 31350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:38589
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.190.keytab
--principal=oryx/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:38589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:56.400333 31350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:56.401360 31350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:56.410715 31359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:56.410715 31356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:56.410948 31357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:56.411175 31350 server_base.cc:1034] running on GCE node
I20260811 04:41:56.411868 31350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:56.413165 31350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:56.414373 31350 hybrid_clock.cc:648] HybridClock initialized: now 1786423316414340 us; error 50 us; skew 500 ppm
Aug 11 04:41:56 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:56.417634 31350 init.cc:377] Logged in from keytab as oryx/127.22.144.190@KRBTEST.COM (short username oryx)
I20260811 04:41:56.418658 31350 webserver.cc:460] Webserver started at http://127.22.144.190:34675/ using document root <none> and password file <none>
I20260811 04:41:56.419354 31350 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:56.419466 31350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:56.419737 31350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:56.422204 31350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "d00de4dd58f249cd996008ba055669e7"
format_stamp: "Formatted at 2026-08-11 04:41:56 on dist-test-slave-ss4n"
server_key: "b2f4cf3e9e3a5982ad749166d74f7e11"
server_key_iv: "58482a3c4b3002f3148aec5fa74d2d37"
server_key_version: "encryptionkey@0"
I20260811 04:41:56.422845 31350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "d00de4dd58f249cd996008ba055669e7"
format_stamp: "Formatted at 2026-08-11 04:41:56 on dist-test-slave-ss4n"
server_key: "b2f4cf3e9e3a5982ad749166d74f7e11"
server_key_iv: "58482a3c4b3002f3148aec5fa74d2d37"
server_key_version: "encryptionkey@0"
I20260811 04:41:56.427193 31350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:56.430054 31366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:56.431553 31350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:56.431727 31350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "d00de4dd58f249cd996008ba055669e7"
format_stamp: "Formatted at 2026-08-11 04:41:56 on dist-test-slave-ss4n"
server_key: "b2f4cf3e9e3a5982ad749166d74f7e11"
server_key_iv: "58482a3c4b3002f3148aec5fa74d2d37"
server_key_version: "encryptionkey@0"
I20260811 04:41:56.431862 31350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:56.471189 31350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:56.475073 31350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:56.475324 31350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:56.484063 31350 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:38589
I20260811 04:41:56.484093 31434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:38589 every 8 connection(s)
I20260811 04:41:56.485108 31350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:41:56.487905 31435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:56.493582 31435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Bootstrap starting.
I20260811 04:41:56.494607 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 31350
I20260811 04:41:56.494750 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:41:56.495077 23106 external_mini_cluster.cc:1428] Setting key 98dee514b41073a8875ebb4cfd65543b
I20260811 04:41:56.496654 31435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:56.497444 31435 log.cc:824] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:56.499650 31435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: No bootstrap required, opened a new log
I20260811 04:41:56.503325 31435 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00de4dd58f249cd996008ba055669e7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38589 } }
I20260811 04:41:56.503600 31435 raft_consensus.cc:374] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:56.503687 31435 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00de4dd58f249cd996008ba055669e7, State: Initialized, Role: FOLLOWER
I20260811 04:41:56.504166 31435 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00de4dd58f249cd996008ba055669e7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38589 } }
I20260811 04:41:56.504324 31435 raft_consensus.cc:388] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:41:56.504415 31435 raft_consensus.cc:482] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:41:56.504540 31435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 04:41:56 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:56.505925 31435 raft_consensus.cc:504] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00de4dd58f249cd996008ba055669e7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38589 } }
I20260811 04:41:56.506335 31435 leader_election.cc:302] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d00de4dd58f249cd996008ba055669e7; no voters:
I20260811 04:41:56.506723 31435 leader_election.cc:288] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:41:56.506865 31441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:56.507145 31441 raft_consensus.cc:686] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [term 1 LEADER]: Becoming Leader. State: Replica: d00de4dd58f249cd996008ba055669e7, State: Running, Role: LEADER
I20260811 04:41:56.507512 31441 consensus_queue.cc:237] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00de4dd58f249cd996008ba055669e7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38589 } }
I20260811 04:41:56.508093 31435 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:41:56.508986 31444 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d00de4dd58f249cd996008ba055669e7. Latest consensus state: current_term: 1 leader_uuid: "d00de4dd58f249cd996008ba055669e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00de4dd58f249cd996008ba055669e7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38589 } } }
I20260811 04:41:56.509009 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d00de4dd58f249cd996008ba055669e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00de4dd58f249cd996008ba055669e7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38589 } } }
I20260811 04:41:56.509155 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:56.509155 31444 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:41:56.509661 31453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:41:56.512398 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:56.496302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.22.144.190:38589)
0811 04:41:56.496853 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:56.496870 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:56.496909 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:41:56.497736 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:56.497758 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:56.498222 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:56.498780 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:56.499720 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.500875 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.501921 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.502498 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.502709 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:56.505941 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:56.505948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:56.505962 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:56.505994 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:56.508272 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:56.508836 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:56.508842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:56.508850 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:56.509018 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:56.509558 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:56.509562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:56.509566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:56.510155 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:56.510406 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:56.511064 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 04:41:56.511385 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":142,"threads_started":1}
I20260811 04:41:56.513343 31453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:41:56.520304 31453 catalog_manager.cc:1347] Generated new cluster ID: a801d822d53342e38cfb93ce77d9d078
I20260811 04:41:56.520392 31453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:41:56.534565 31453 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:41:56.535815 31453 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:41:56.545112 31453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Generated new TSK 0
I20260811 04:41:56.546087 31453 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab.
Entry for principal oryx/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab.
I20260811 04:41:56.585034 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:38589
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:56.705848 31466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:56.706140 31466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:56.706194 31466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:56.710608 31466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:56.710693 31466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:56.710736 31466 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:56.710770 31466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:41:56.716344 31466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:38589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:56.717458 31466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:56.718518 31466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:56.729086 31475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:56.729163 31473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:56.729163 31472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:56.729430 31466 server_base.cc:1034] running on GCE node
I20260811 04:41:56.729899 31466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:56.730660 31466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:56.731873 31466 hybrid_clock.cc:648] HybridClock initialized: now 1786423316731848 us; error 38 us; skew 500 ppm
Aug 11 04:41:56 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:56.738749 31466 init.cc:377] Logged in from keytab as oryx/127.22.144.129@KRBTEST.COM (short username oryx)
I20260811 04:41:56.739867 31466 webserver.cc:460] Webserver started at http://127.22.144.129:46843/ using document root <none> and password file <none>
I20260811 04:41:56.740526 31466 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:56.740636 31466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:56.740908 31466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:56.743374 31466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2"
format_stamp: "Formatted at 2026-08-11 04:41:56 on dist-test-slave-ss4n"
server_key: "1a290dc31570eb29459aee1ca49b23c0"
server_key_iv: "9f06bb5ec6ac570563b612273df2eb28"
server_key_version: "encryptionkey@0"
I20260811 04:41:56.744050 31466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2"
format_stamp: "Formatted at 2026-08-11 04:41:56 on dist-test-slave-ss4n"
server_key: "1a290dc31570eb29459aee1ca49b23c0"
server_key_iv: "9f06bb5ec6ac570563b612273df2eb28"
server_key_version: "encryptionkey@0"
I20260811 04:41:56.748798 31466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:41:56.751854 31483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:56.753285 31466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:56.753537 31466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2"
format_stamp: "Formatted at 2026-08-11 04:41:56 on dist-test-slave-ss4n"
server_key: "1a290dc31570eb29459aee1ca49b23c0"
server_key_iv: "9f06bb5ec6ac570563b612273df2eb28"
server_key_version: "encryptionkey@0"
I20260811 04:41:56.753697 31466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:56.781777 31466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:56.785624 31466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:56.785854 31466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:56.787443 31466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:56.787539 31466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:56.787616 31466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:56.787676 31466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:56.800177 31466 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:42365
I20260811 04:41:56.800379 31609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:42365 every 8 connection(s)
Aug 11 04:41:56 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:56.801584 31466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:41:56.803562 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 31466
I20260811 04:41:56.803730 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:41:56.804132 23106 external_mini_cluster.cc:1428] Setting key 300327e93f5ac1036fb0c4368eb109ea
I20260811 04:41:56.807768 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:56.788477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:48957 (local address 127.22.144.190:38589)
0811 04:41:56.788677 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:56.788681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:56.789904 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 04:41:56.791086 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:56.791092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:56.791166 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:56.791257 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:56.793274 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.794316 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.797435 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.797891 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.798048 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:56.802304 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:56.802308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:56.802311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:56.802346 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:56.804336 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:56.805577 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:56.805581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:56.805583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:56.805679 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:56.806127 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:56.806133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:56.806135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:56.806424 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:56.806555 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:56.807360 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 04:41:56.807527 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 04:41:56.809082 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:56.789101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.22.144.129:48957)
0811 04:41:56.789738 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:56.789787 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:56.790792 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:56.791768 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:56.791780 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:56.792293 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:56.793084 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:56.793111 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.794457 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.794459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:56.797283 (+ 2824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:56.797296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.798234 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.798240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:56.798499 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:56.799309 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:56.799335 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:56.802020 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:56.804486 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:56.804495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:56.804515 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:56.804987 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:56.805809 (+ 822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:56.805814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:56.805816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:56.806026 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:56.806637 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:56.806645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:56.807019 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 04:41:56.807282 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 04:41:56.807704 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":158,"threads_started":1}
I20260811 04:41:56.810786 31610 heartbeater.cc:347] Connected to a master server at 127.22.144.190:38589
I20260811 04:41:56.810982 31610 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:56.811712 31610 heartbeater.cc:511] Master 127.22.144.190:38589 requested a full tablet report, sending...
I20260811 04:41:56.814016 31381 ts_manager.cc:194] Registered new tserver with Master: 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365)
I20260811 04:41:56.815502 31381 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:48957
WARNING: no policy specified for oryx/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.130@KRBTEST.COM" created.
I20260811 04:41:56.826193 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:56.813678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:43945 (local address 127.22.144.190:38589)
0811 04:41:56.813821 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:56.813827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:56.815937 (+ 2110us) server_negotiation.cc:407] Connection header received
0811 04:41:56.816091 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:56.816095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:56.816145 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:56.816236 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:56.816961 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.817848 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.819219 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.819737 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.819861 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:56.821373 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:56.821380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:56.821385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:56.821422 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:56.823724 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:56.824368 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:56.824373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:56.824375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:56.824497 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:56.824892 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:56.824897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:56.824899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:56.825206 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:56.825316 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:56.825747 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 04:41:56.825928 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 04:41:56.826943 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:56.813263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.22.144.129:43945)
0811 04:41:56.815817 (+ 2554us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:56.815836 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:56.815974 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:56.816373 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:56.816378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:56.816658 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:56.816807 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:56.816816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.818035 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.818050 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:56.818975 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:56.818988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:56.819894 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:56.819900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:56.820020 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:56.820683 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:56.820701 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:56.821200 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:56.823858 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:56.823861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:56.823864 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:56.824244 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:56.824634 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:56.824637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:56.824640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:56.824781 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:56.825315 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:56.825319 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:56.825444 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 04:41:56.826462 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0811 04:41:56.826671 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2425,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab.
Entry for principal oryx/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab.
I20260811 04:41:56.846417 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:38589
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:56.977545 31620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:56.977869 31620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:56.977932 31620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:56.982031 31620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:56.982095 31620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:56.982134 31620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:56.982187 31620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:41:56.987668 31620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:38589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:56.988713 31620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:56.989781 31620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:56.999758 31626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:56.999811 31629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:56.999768 31627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:57.000161 31620 server_base.cc:1034] running on GCE node
I20260811 04:41:57.000626 31620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:57.001276 31620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:57.002475 31620 hybrid_clock.cc:648] HybridClock initialized: now 1786423317002462 us; error 60 us; skew 500 ppm
Aug 11 04:41:56 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:57.005833 31620 init.cc:377] Logged in from keytab as oryx/127.22.144.130@KRBTEST.COM (short username oryx)
I20260811 04:41:57.006821 31620 webserver.cc:460] Webserver started at http://127.22.144.130:46117/ using document root <none> and password file <none>
I20260811 04:41:57.007438 31620 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:57.007514 31620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:57.007820 31620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:57.010356 31620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "3a7b935433e1456a87188e5cbe0c588d"
format_stamp: "Formatted at 2026-08-11 04:41:57 on dist-test-slave-ss4n"
server_key: "197aac6630962c2dcff461d3cf3cd924"
server_key_iv: "b7f5fa629fe066f34bc2a45fc52a2f5f"
server_key_version: "encryptionkey@0"
I20260811 04:41:57.011143 31620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "3a7b935433e1456a87188e5cbe0c588d"
format_stamp: "Formatted at 2026-08-11 04:41:57 on dist-test-slave-ss4n"
server_key: "197aac6630962c2dcff461d3cf3cd924"
server_key_iv: "b7f5fa629fe066f34bc2a45fc52a2f5f"
server_key_version: "encryptionkey@0"
I20260811 04:41:57.015465 31620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:41:57.018395 31636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:57.019922 31620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:57.020066 31620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "3a7b935433e1456a87188e5cbe0c588d"
format_stamp: "Formatted at 2026-08-11 04:41:57 on dist-test-slave-ss4n"
server_key: "197aac6630962c2dcff461d3cf3cd924"
server_key_iv: "b7f5fa629fe066f34bc2a45fc52a2f5f"
server_key_version: "encryptionkey@0"
I20260811 04:41:57.020192 31620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:57.038545 31620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:57.042222 31620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:57.042481 31620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:57.044215 31620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:57.044306 31620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:57.044387 31620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:57.044412 31620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:57.056914 31620 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:34121
I20260811 04:41:57.057087 31770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:34121 every 8 connection(s)
I20260811 04:41:57.058090 31620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:41:57.062011 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.045836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:45459 (local address 127.22.144.190:38589)
0811 04:41:57.045993 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.045997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.047338 (+ 1341us) server_negotiation.cc:407] Connection header received
0811 04:41:57.048819 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.048823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.048882 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.049001 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.050802 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.051344 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.053180 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.053417 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.053911 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.057573 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.057577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.057580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.057630 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.059701 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.060283 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.060287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.060288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.060380 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.060738 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.060741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.060742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.060958 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.061049 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.061753 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.061874 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 04:41:57.062969 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.046437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.22.144.130:45459)
0811 04:41:57.047135 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.047204 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.048435 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.049197 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.049209 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.049826 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:57.050629 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.050667 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.051492 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.051495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.053042 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.053058 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.053947 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.053953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.054234 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.055114 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:57.055166 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:57.057290 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:57.059843 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.059848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.059860 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.060174 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.060485 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.060488 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.060489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.060637 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.061060 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:57.061077 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:57.061337 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.061698 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.061995 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":160,"threads_started":1}
I20260811 04:41:57.064441 31771 heartbeater.cc:347] Connected to a master server at 127.22.144.190:38589
I20260811 04:41:57.064673 31771 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:57.065393 31771 heartbeater.cc:511] Master 127.22.144.190:38589 requested a full tablet report, sending...
I20260811 04:41:57.066105 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 31620
I20260811 04:41:57.066231 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:41:57.066579 23106 external_mini_cluster.cc:1428] Setting key 3350864c1abc0607e5de4bf9e516f30e
I20260811 04:41:57.066843 31384 ts_manager.cc:194] Registered new tserver with Master: 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:57.067770 31384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:45459
I20260811 04:41:57.076997 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.067304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:54657 (local address 127.22.144.190:38589)
0811 04:41:57.067419 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.067423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.067741 (+ 318us) server_negotiation.cc:407] Connection header received
0811 04:41:57.068054 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.068059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.068109 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.068219 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.068842 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.069710 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.070870 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.071139 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.071265 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.072844 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.072847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.072849 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.072873 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.074784 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.075387 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.075390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.075392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.075478 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.075893 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.075895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.075897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.076184 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.076280 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.076702 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.076823 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 04:41:57.077080 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.067329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.22.144.130:54657)
0811 04:41:57.067657 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.067673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.067835 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.068240 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.068244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.068511 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:57.068682 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.068692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.069893 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.069897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.070729 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.070740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.071273 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.071277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.071386 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.072051 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:57.072074 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:57.072684 (+ 610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:57.074921 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.074928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.074934 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.075269 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.075597 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.075603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.075607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.075743 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.076309 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:57.076315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:57.076457 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.076610 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.076806 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":2,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for oryx/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab.
Entry for principal oryx/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab.
I20260811 04:41:57.102646 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:38589
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:57.223903 31780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:57.224233 31780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:57.224329 31780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:57.228600 31780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:57.228700 31780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:57.228780 31780 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 04:41:57.228861 31780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:41:57.234413 31780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.144.190:38589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:57.235646 31780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:57.236827 31780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:57.247440 31791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:57.247459 31786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:57.247440 31785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:57.247961 31780 server_base.cc:1034] running on GCE node
I20260811 04:41:57.248368 31780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:57.249055 31780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:57.250252 31780 hybrid_clock.cc:648] HybridClock initialized: now 1786423317250229 us; error 35 us; skew 500 ppm
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423317, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:41:57.253290 31780 init.cc:377] Logged in from keytab as oryx/127.22.144.131@KRBTEST.COM (short username oryx)
I20260811 04:41:57.254406 31780 webserver.cc:460] Webserver started at http://127.22.144.131:33217/ using document root <none> and password file <none>
I20260811 04:41:57.254947 31780 fs_manager.cc:359] Metadata directory not provided
I20260811 04:41:57.255009 31780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:41:57.255194 31780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:41:57.257330 31780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "ff37bb35f9934c85b6850ee5ae0515c3"
format_stamp: "Formatted at 2026-08-11 04:41:57 on dist-test-slave-ss4n"
server_key: "71ffe2aa414b9e572d78d3d6c5d417be"
server_key_iv: "15dc149f946f264705b0b123a3dc0d8e"
server_key_version: "encryptionkey@0"
I20260811 04:41:57.257987 31780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "ff37bb35f9934c85b6850ee5ae0515c3"
format_stamp: "Formatted at 2026-08-11 04:41:57 on dist-test-slave-ss4n"
server_key: "71ffe2aa414b9e572d78d3d6c5d417be"
server_key_iv: "15dc149f946f264705b0b123a3dc0d8e"
server_key_version: "encryptionkey@0"
I20260811 04:41:57.261917 31780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:41:57.264631 31799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:57.266285 31780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:41:57.266423 31780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "ff37bb35f9934c85b6850ee5ae0515c3"
format_stamp: "Formatted at 2026-08-11 04:41:57 on dist-test-slave-ss4n"
server_key: "71ffe2aa414b9e572d78d3d6c5d417be"
server_key_iv: "15dc149f946f264705b0b123a3dc0d8e"
server_key_version: "encryptionkey@0"
I20260811 04:41:57.266563 31780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:41:57.289005 31780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:41:57.292608 31780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:41:57.292842 31780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:41:57.294588 31780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:41:57.294643 31780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:41:57.294685 31780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:41:57.294701 31780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423317, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:57.306171 31780 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:45725
I20260811 04:41:57.306243 31926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:45725 every 8 connection(s)
I20260811 04:41:57.307380 31780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:41:57.311299 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.295604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:57429 (local address 127.22.144.190:38589)
0811 04:41:57.295776 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.295782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.296710 (+ 928us) server_negotiation.cc:407] Connection header received
0811 04:41:57.298101 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.298106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.298176 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.298294 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.299747 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.300581 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.302518 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.303043 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.303196 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.306291 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.306295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.306298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.306329 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.308251 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.309589 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.309592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.309594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.309685 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.310068 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.310073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.310074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.310281 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.310393 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.310977 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.311121 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 04:41:57.311798 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 31780
I20260811 04:41:57.311975 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:41:57.312271 23106 external_mini_cluster.cc:1428] Setting key 5bd5c8806b61b47d0752f9fceffe3d94
I20260811 04:41:57.312321 31809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.296009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.22.144.131:57429)
0811 04:41:57.296519 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.296574 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.297755 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.298500 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.298507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.298874 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:57.299556 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.299574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.300784 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.300790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.302357 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.302370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.303244 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.303250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.303453 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.304162 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:57.304189 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:57.306049 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:57.308364 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.308370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.308386 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.308840 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.309787 (+ 947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.309791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.309793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.309948 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.310394 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:57.310402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:57.310696 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.310916 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.311253 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":150,"threads_started":1}
I20260811 04:41:57.313892 31927 heartbeater.cc:347] Connected to a master server at 127.22.144.190:38589
I20260811 04:41:57.314066 31927 heartbeater.cc:464] Registering TS with master...
I20260811 04:41:57.314620 31927 heartbeater.cc:511] Master 127.22.144.190:38589 requested a full tablet report, sending...
I20260811 04:41:57.315708 31382 ts_manager.cc:194] Registered new tserver with Master: ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725)
I20260811 04:41:57.316366 31382 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:57429
I20260811 04:41:57.316906 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:41:57.324746 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.316841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:59537 (local address 127.22.144.190:38589)
0811 04:41:57.317283 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.317288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.317305 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:41:57.317346 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.317350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.317405 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.317561 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.318115 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.318970 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.319815 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.320041 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.320151 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.321330 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.321334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.321337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.321370 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.322948 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.323409 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.323413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.323415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.323502 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.323805 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.323810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.323812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.324009 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.324128 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.324456 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.324584 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355}
I20260811 04:41:57.324762 31933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.316681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.22.144.131:59537)
0811 04:41:57.317103 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.317117 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.317224 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.317580 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.317583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.317825 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:57.317943 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.317950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.319120 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.319122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.319684 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.319693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.320169 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.320173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.320276 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.320839 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:57.320851 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:57.321187 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:57.323074 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.323079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.323081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.323298 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.323608 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.323612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.323615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.323709 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.324180 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:57.324183 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:57.324280 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.324395 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.324568 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":130,"threads_started":1}
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 04:41:57.345901 31439 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:57.338834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39826 (local address 127.22.144.190:38589)
0811 04:41:57.338980 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.338984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.338999 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:41:57.339048 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.339051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.339096 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.339183 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.340103 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.340679 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.341810 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.342052 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.342141 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.345706 (+ 3565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39826: BlockingRecv error: recv got EOF from 127.0.0.1:39826 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:57.356696 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.347467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.22.144.190:38589)
0811 04:41:57.347671 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.347674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.347826 (+ 152us) server_negotiation.cc:407] Connection header received
0811 04:41:57.347941 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.347943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.347993 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.348075 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.348706 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.349215 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.350173 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.350542 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.350677 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.353566 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.353570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.353573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.353612 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.354940 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.355411 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.355415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.355417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.355502 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.355786 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.355790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.355792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.355991 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.356147 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.356391 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.356517 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 04:41:57.360528 31382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:57.364044 31382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:57.380939 31953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.374363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34121 (local address 127.0.0.1:56332)
0811 04:41:57.376092 (+ 1729us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.376105 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.376268 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.376663 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.376665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.376683 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.376768 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.376774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.377810 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.377813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.379210 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.379222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.380328 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.380332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.380462 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.380504 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.380638 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.380765 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1670,"mutex_wait_us":134,"thread_start_us":97,"threads_started":1}
I20260811 04:41:57.380955 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.374698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56332 (local address 127.22.144.130:34121)
0811 04:41:57.375116 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.375129 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.376196 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 04:41:57.376385 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.376390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.376527 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.376689 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.376866 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.377693 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.379348 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.380192 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.380336 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.380474 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.380690 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.380795 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":113,"threads_started":1}
I20260811 04:41:57.381464 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.374004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:45725 (local address 127.0.0.1:54224)
0811 04:41:57.374938 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.374995 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.375144 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.375884 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.375889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.375911 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.376020 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.376025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.377593 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.377596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.379216 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.379222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.380921 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.380925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.381038 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.381143 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.381227 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.381304 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":135,"spinlock_wait_cycles":4736,"thread_start_us":106,"threads_started":1}
I20260811 04:41:57.380986 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.374066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42365 (local address 127.0.0.1:34950)
0811 04:41:57.374937 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.374995 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.375146 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.376006 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.376010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.376041 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.376147 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.376157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.378659 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.378664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.379585 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.379596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.380650 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.380653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.380735 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.380756 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.380820 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.380873 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":3,"spinlock_wait_cycles":2816,"thread_start_us":88,"threads_started":1}
I20260811 04:41:57.381862 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.374114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.22.144.131:45725)
0811 04:41:57.375328 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.375339 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.375362 (+ 23us) server_negotiation.cc:407] Connection header received
0811 04:41:57.375460 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.375466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.375689 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.375869 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.376152 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.377404 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.379762 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.380569 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.380738 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.380854 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.381568 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.381683 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":139,"threads_started":1}
I20260811 04:41:57.382079 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.374864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.22.144.129:42365)
0811 04:41:57.375442 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.375448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.375467 (+ 19us) server_negotiation.cc:407] Connection header received
0811 04:41:57.375531 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.375536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.375689 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.375847 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.376281 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.377403 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.379731 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.380327 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.380456 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.380586 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.381240 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.381347 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":89,"threads_started":1}
I20260811 04:41:57.384627 31849 tablet_service.cc:1467] Processing CreateTablet for tablet d86fda62bea941cd9132bbfbc1c537dd (DEFAULT_TABLE table=test-table [id=93c086325cdd491090c9d14e4bac6657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:57.384614 31692 tablet_service.cc:1467] Processing CreateTablet for tablet d86fda62bea941cd9132bbfbc1c537dd (DEFAULT_TABLE table=test-table [id=93c086325cdd491090c9d14e4bac6657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:57.384689 31535 tablet_service.cc:1467] Processing CreateTablet for tablet d86fda62bea941cd9132bbfbc1c537dd (DEFAULT_TABLE table=test-table [id=93c086325cdd491090c9d14e4bac6657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:57.385793 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d86fda62bea941cd9132bbfbc1c537dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.385793 31692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d86fda62bea941cd9132bbfbc1c537dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.385794 31849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d86fda62bea941cd9132bbfbc1c537dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.392560 31958 tablet_bootstrap.cc:492] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Bootstrap starting.
I20260811 04:41:57.394286 31959 tablet_bootstrap.cc:492] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Bootstrap starting.
I20260811 04:41:57.395025 31958 tablet_bootstrap.cc:654] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.395839 31958 log.cc:824] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:57.396301 31959 tablet_bootstrap.cc:654] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.396592 31960 tablet_bootstrap.cc:492] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Bootstrap starting.
I20260811 04:41:57.397040 31959 log.cc:824] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:57.398492 31960 tablet_bootstrap.cc:654] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.398850 31959 tablet_bootstrap.cc:492] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: No bootstrap required, opened a new log
I20260811 04:41:57.398918 31958 tablet_bootstrap.cc:492] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: No bootstrap required, opened a new log
I20260811 04:41:57.399065 31959 ts_tablet_manager.cc:1397] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 04:41:57.399127 31958 ts_tablet_manager.cc:1397] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 04:41:57.399291 31960 log.cc:824] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Log is configured to *not* fsync() on all Append() calls
I20260811 04:41:57.401467 31960 tablet_bootstrap.cc:492] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: No bootstrap required, opened a new log
I20260811 04:41:57.401708 31960 ts_tablet_manager.cc:1397] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 04:41:57.402644 31959 raft_consensus.cc:348] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.402889 31959 raft_consensus.cc:374] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.402951 31959 raft_consensus.cc:729] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7b935433e1456a87188e5cbe0c588d, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.403103 31958 raft_consensus.cc:348] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.403405 31958 raft_consensus.cc:374] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.403461 31958 raft_consensus.cc:729] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.403453 31959 consensus_queue.cc:260] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.403924 31958 consensus_queue.cc:260] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.404124 31771 heartbeater.cc:503] Master 127.22.144.190:38589 was elected leader, sending a full tablet report...
I20260811 04:41:57.404525 31959 ts_tablet_manager.cc:1428] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 04:41:57.404873 31958 ts_tablet_manager.cc:1428] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:41:57.405206 31927 heartbeater.cc:503] Master 127.22.144.190:38589 was elected leader, sending a full tablet report...
I20260811 04:41:57.406963 31960 raft_consensus.cc:348] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.407188 31960 raft_consensus.cc:374] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.407248 31960 raft_consensus.cc:729] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.407712 31960 consensus_queue.cc:260] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.408375 31610 heartbeater.cc:503] Master 127.22.144.190:38589 was elected leader, sending a full tablet report...
I20260811 04:41:57.408730 31960 ts_tablet_manager.cc:1428] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260811 04:41:57.434082 31969 raft_consensus.cc:482] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:57.434399 31969 raft_consensus.cc:504] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.435990 31969 leader_election.cc:288] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:57.440742 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.436688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:58945 (local address 127.22.144.130:34121)
0811 04:41:57.436850 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.436856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.437074 (+ 218us) server_negotiation.cc:407] Connection header received
0811 04:41:57.437212 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.437215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.437265 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.437359 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.437731 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.438405 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.439677 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.440093 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.440229 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.440286 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.440488 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.440566 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 04:41:57.440912 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.436361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53107 (local address 127.22.144.131:45725)
0811 04:41:57.436541 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.436546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.436748 (+ 202us) server_negotiation.cc:407] Connection header received
0811 04:41:57.436919 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.436922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.436978 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.437070 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.437504 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.438127 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.439799 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.440254 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.440406 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.440464 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.440708 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.440795 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:41:57.440980 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.436234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:45725 (local address 127.22.144.129:53107)
0811 04:41:57.436621 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.436639 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.436782 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.437156 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.437163 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.437187 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.437346 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.437364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.438262 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.438266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.439583 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.439595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.440400 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.440404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.440531 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.440561 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.440670 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.440748 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":42,"thread_start_us":175,"threads_started":1}
I20260811 04:41:57.441256 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.436347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34121 (local address 127.22.144.129:58945)
0811 04:41:57.436959 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.436974 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.437093 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.437384 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.437386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.437403 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.437568 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.437577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.438534 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.438536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.439547 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.439557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.440225 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.440229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.440328 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.440364 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.440960 (+ 596us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.441036 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":1,"thread_start_us":132,"threads_started":1}
I20260811 04:41:57.441823 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d86fda62bea941cd9132bbfbc1c537dd" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" is_pre_election: true
I20260811 04:41:57.441875 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d86fda62bea941cd9132bbfbc1c537dd" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d" is_pre_election: true
I20260811 04:41:57.442140 31875 raft_consensus.cc:2454] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 0.
I20260811 04:41:57.442193 31716 raft_consensus.cc:2454] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 0.
I20260811 04:41:57.442684 31484 leader_election.cc:302] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d7686fd146d4aa3a30f8c3fc0a46fb2, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:57.442932 31969 raft_consensus.cc:2781] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:57.443040 31969 raft_consensus.cc:482] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:57.443101 31969 raft_consensus.cc:3037] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.444262 31969 raft_consensus.cc:504] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.444655 31969 leader_election.cc:288] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 election: Requested vote from peers ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:57.445010 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d86fda62bea941cd9132bbfbc1c537dd" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d"
I20260811 04:41:57.445043 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d86fda62bea941cd9132bbfbc1c537dd" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff37bb35f9934c85b6850ee5ae0515c3"
I20260811 04:41:57.445155 31875 raft_consensus.cc:3037] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.445156 31716 raft_consensus.cc:3037] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.446501 31716 raft_consensus.cc:2454] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 1.
I20260811 04:41:57.446501 31875 raft_consensus.cc:2454] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 1.
I20260811 04:41:57.446843 31484 leader_election.cc:302] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d7686fd146d4aa3a30f8c3fc0a46fb2, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:57.447116 31969 raft_consensus.cc:2781] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:57.447363 31969 raft_consensus.cc:686] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Running, Role: LEADER
I20260811 04:41:57.447708 31969 consensus_queue.cc:237] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } }
I20260811 04:41:57.451148 31382 catalog_manager.cc:5495] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129). New cstate: current_term: 1 leader_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } health_report { overall_health: UNKNOWN } } }
I20260811 04:41:57.464715 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.461036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.22.144.129:42365)
0811 04:41:57.461184 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.461189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.461205 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:57.461326 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.461328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.461380 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.461490 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:57.461771 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.462321 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.463207 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.463428 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.463580 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.463867 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.464301 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.464431 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.464498 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.464566 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 04:41:57.471655 31716 raft_consensus.cc:1261] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Refusing update from remote peer 4d7686fd146d4aa3a30f8c3fc0a46fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:57.471889 31875 raft_consensus.cc:1261] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d7686fd146d4aa3a30f8c3fc0a46fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:57.472472 31969 consensus_queue.cc:1035] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:57.472646 31972 consensus_queue.cc:1035] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:57.482026 31977 mvcc.cc:205] Tried to move back new op lower bound from 7317189908353646592 to 7317189908268322816. Current Snapshot: MvccSnapshot[applied={T|T < 7317189908353646592}]
I20260811 04:41:57.490717 31381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:57.490899 31381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:57.493294 31381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Sending DeleteTablet for 3 replicas of tablet d86fda62bea941cd9132bbfbc1c537dd
I20260811 04:41:57.494221 31849 tablet_service.cc:1514] Processing DeleteTablet for tablet d86fda62bea941cd9132bbfbc1c537dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:57 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54224
I20260811 04:41:57.494397 31692 tablet_service.cc:1514] Processing DeleteTablet for tablet d86fda62bea941cd9132bbfbc1c537dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:57 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:56332
I20260811 04:41:57.494899 31535 tablet_service.cc:1514] Processing DeleteTablet for tablet d86fda62bea941cd9132bbfbc1c537dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:57 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:34950
I20260811 04:41:57.495076 31381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 04:41:57.495532 31990 tablet_replica.cc:330] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: stopping tablet replica
W20260811 04:41:57.495534 31381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:57.495961 31990 raft_consensus.cc:2229] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:57.496423 31990 raft_consensus.cc:2258] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:57.494925 31989 tablet_replica.cc:330] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: stopping tablet replica
I20260811 04:41:57.500499 31535 tablet_service.cc:1467] Processing CreateTablet for tablet 73a7592f1808495badea78d1033a46fa (DEFAULT_TABLE table=test-table [id=7e304348d2c9480abe4a0a159b7f7414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:57.500806 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73a7592f1808495badea78d1033a46fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.501127 31849 tablet_service.cc:1467] Processing CreateTablet for tablet 73a7592f1808495badea78d1033a46fa (DEFAULT_TABLE table=test-table [id=7e304348d2c9480abe4a0a159b7f7414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:57.499336 31989 raft_consensus.cc:2229] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:57.501451 31849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73a7592f1808495badea78d1033a46fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.501847 31989 raft_consensus.cc:2258] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:57.503701 31692 tablet_service.cc:1467] Processing CreateTablet for tablet 73a7592f1808495badea78d1033a46fa (DEFAULT_TABLE table=test-table [id=7e304348d2c9480abe4a0a159b7f7414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:57.503953 31692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73a7592f1808495badea78d1033a46fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.504315 31958 tablet_bootstrap.cc:492] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Bootstrap starting.
I20260811 04:41:57.505262 31958 tablet_bootstrap.cc:654] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.506269 31989 ts_tablet_manager.cc:1905] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:57.506317 31959 tablet_bootstrap.cc:492] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Bootstrap starting.
I20260811 04:41:57.507524 31959 tablet_bootstrap.cc:654] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.508180 31960 tablet_bootstrap.cc:492] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Bootstrap starting.
I20260811 04:41:57.508857 31959 tablet_bootstrap.cc:492] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: No bootstrap required, opened a new log
I20260811 04:41:57.508949 31959 ts_tablet_manager.cc:1397] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:57.509151 31960 tablet_bootstrap.cc:654] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.509394 31989 ts_tablet_manager.cc:1918] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:57.509536 31989 log.cc:1196] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/d86fda62bea941cd9132bbfbc1c537dd
I20260811 04:41:57.509652 31988 tablet_replica.cc:330] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: stopping tablet replica
I20260811 04:41:57.509634 31959 raft_consensus.cc:348] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.509773 31959 raft_consensus.cc:374] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.509810 31959 raft_consensus.cc:729] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7b935433e1456a87188e5cbe0c588d, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.509914 31989 ts_tablet_manager.cc:1939] T d86fda62bea941cd9132bbfbc1c537dd P 3a7b935433e1456a87188e5cbe0c588d: Deleting consensus metadata
I20260811 04:41:57.510025 31988 raft_consensus.cc:2229] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:57.510366 31988 raft_consensus.cc:2258] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:57.511349 31369 catalog_manager.cc:4841] TS 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121): tablet d86fda62bea941cd9132bbfbc1c537dd (table test-table [id=93c086325cdd491090c9d14e4bac6657]) successfully deleted
I20260811 04:41:57.511854 31990 ts_tablet_manager.cc:1905] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:57.509965 31959 consensus_queue.cc:260] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.512408 31959 ts_tablet_manager.cc:1428] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:57.512493 31960 tablet_bootstrap.cc:492] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: No bootstrap required, opened a new log
I20260811 04:41:57.512523 31958 tablet_bootstrap.cc:492] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: No bootstrap required, opened a new log
I20260811 04:41:57.512617 31960 ts_tablet_manager.cc:1397] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 04:41:57.512620 31958 ts_tablet_manager.cc:1397] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260811 04:41:57.513087 31958 raft_consensus.cc:348] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.513091 31960 raft_consensus.cc:348] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.513226 31958 raft_consensus.cc:374] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.513223 31960 raft_consensus.cc:374] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.513274 31958 raft_consensus.cc:729] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.513279 31960 raft_consensus.cc:729] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.513414 31960 consensus_queue.cc:260] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.513432 31958 consensus_queue.cc:260] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.513729 31958 ts_tablet_manager.cc:1428] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 04:41:57.513736 31960 ts_tablet_manager.cc:1428] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:57.515058 31990 ts_tablet_manager.cc:1918] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:57.515161 31990 log.cc:1196] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/d86fda62bea941cd9132bbfbc1c537dd
I20260811 04:41:57.515534 31990 ts_tablet_manager.cc:1939] T d86fda62bea941cd9132bbfbc1c537dd P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting consensus metadata
I20260811 04:41:57.515727 31988 ts_tablet_manager.cc:1905] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:57.516745 31370 catalog_manager.cc:4841] TS 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365): tablet d86fda62bea941cd9132bbfbc1c537dd (table test-table [id=93c086325cdd491090c9d14e4bac6657]) successfully deleted
I20260811 04:41:57.518533 31988 ts_tablet_manager.cc:1918] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:57.518666 31988 log.cc:1196] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/d86fda62bea941cd9132bbfbc1c537dd
I20260811 04:41:57.518998 31988 ts_tablet_manager.cc:1939] T d86fda62bea941cd9132bbfbc1c537dd P ff37bb35f9934c85b6850ee5ae0515c3: Deleting consensus metadata
I20260811 04:41:57.520144 31367 catalog_manager.cc:4841] TS ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725): tablet d86fda62bea941cd9132bbfbc1c537dd (table test-table [id=93c086325cdd491090c9d14e4bac6657]) successfully deleted
W20260811 04:41:57.553182 31611 tablet.cc:2369] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:41:57.558538 31928 tablet.cc:2369] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:41:57.559791 31772 tablet.cc:2369] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:41:57.595780 31968 raft_consensus.cc:482] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:57.595964 31968 raft_consensus.cc:504] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.597265 31968 leader_election.cc:288] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:57.597863 31981 raft_consensus.cc:482] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:57.598013 31981 raft_consensus.cc:504] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.598660 31981 leader_election.cc:288] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121), ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725)
I20260811 04:41:57.599076 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a7592f1808495badea78d1033a46fa" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d" is_pre_election: true
I20260811 04:41:57.599187 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a7592f1808495badea78d1033a46fa" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" is_pre_election: true
I20260811 04:41:57.599262 31716 raft_consensus.cc:2454] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 0.
I20260811 04:41:57.599350 31875 raft_consensus.cc:2454] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 0.
I20260811 04:41:57.599725 31484 leader_election.cc:302] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d7686fd146d4aa3a30f8c3fc0a46fb2, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:57.599943 31981 raft_consensus.cc:2781] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:57.600000 31981 raft_consensus.cc:482] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:57.600045 31981 raft_consensus.cc:3037] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.601678 31981 raft_consensus.cc:504] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.602113 31981 leader_election.cc:288] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 election: Requested vote from peers 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121), ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725)
I20260811 04:41:57.602536 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a7592f1808495badea78d1033a46fa" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d"
I20260811 04:41:57.602665 31716 raft_consensus.cc:3037] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.602648 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a7592f1808495badea78d1033a46fa" candidate_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff37bb35f9934c85b6850ee5ae0515c3"
I20260811 04:41:57.602780 31875 raft_consensus.cc:3037] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.603853 31716 raft_consensus.cc:2454] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 1.
I20260811 04:41:57.604131 31875 raft_consensus.cc:2454] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in term 1.
I20260811 04:41:57.604199 31486 leader_election.cc:302] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7b935433e1456a87188e5cbe0c588d, 4d7686fd146d4aa3a30f8c3fc0a46fb2; no voters:
I20260811 04:41:57.604259 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.597705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:45537 (local address 127.22.144.129:42365)
0811 04:41:57.598304 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.598310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.598578 (+ 268us) server_negotiation.cc:407] Connection header received
0811 04:41:57.598998 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.599003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.599063 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.599127 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.599624 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.600498 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.602576 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.603294 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.603410 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.603488 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.603984 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.604102 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494}
I20260811 04:41:57.604787 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.597887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42365 (local address 127.22.144.131:45537)
0811 04:41:57.598473 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.598492 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.598825 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.599288 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.599291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.599311 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.599462 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.599471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.600669 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.600673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.602413 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.602426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.603624 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.603629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.603767 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.603805 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.604504 (+ 699us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.604599 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":105,"threads_started":1}
I20260811 04:41:57.605185 31981 raft_consensus.cc:2781] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:57.605280 31981 raft_consensus.cc:686] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Running, Role: LEADER
I20260811 04:41:57.605379 31550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a7592f1808495badea78d1033a46fa" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" is_pre_election: true
I20260811 04:41:57.605443 31981 consensus_queue.cc:237] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.606608 31809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.597628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34121 (local address 127.22.144.131:44733)
0811 04:41:57.597798 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.597814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.597922 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.598550 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.598554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.598572 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.598691 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.598697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.600623 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.600627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.602043 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.602055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.606095 (+ 4040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.606101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.606226 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.606257 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.606356 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.606464 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":76}
I20260811 04:41:57.606608 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.598103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:44733 (local address 127.22.144.130:34121)
0811 04:41:57.598231 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.598235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.598252 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:41:57.598306 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.598310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.598357 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.598430 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.599452 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.600230 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.604704 (+ 4474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.605388 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.605584 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.605646 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.606400 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.606485 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:41:57.607128 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73a7592f1808495badea78d1033a46fa" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d" is_pre_election: true
I20260811 04:41:57.607311 31716 raft_consensus.cc:2379] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff37bb35f9934c85b6850ee5ae0515c3 in current term 1: Already voted for candidate 4d7686fd146d4aa3a30f8c3fc0a46fb2 in this term.
I20260811 04:41:57.607659 31803 leader_election.cc:302] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff37bb35f9934c85b6850ee5ae0515c3; no voters: 3a7b935433e1456a87188e5cbe0c588d, 4d7686fd146d4aa3a30f8c3fc0a46fb2
I20260811 04:41:57.608167 31968 raft_consensus.cc:2726] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 04:41:57.608333 31384 catalog_manager.cc:5495] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129). New cstate: current_term: 1 leader_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } health_report { overall_health: UNKNOWN } } }
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423316, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.190@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:57.665347 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.655465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:38589 (local address 127.0.0.1:39840)
0811 04:41:57.655809 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.655824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.655965 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.656238 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.656241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.656744 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:41:57.656870 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.656878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.658042 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.658046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.658863 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.658875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.659384 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.659387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.659476 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.660114 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:41:57.660128 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:41:57.661803 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:41:57.663545 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.663554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.663559 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.663812 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.664120 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:41:57.664123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:41:57.664124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:41:57.664209 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:41:57.664685 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:41:57.664692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:41:57.664815 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.664945 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.665167 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":136,"threads_started":1}
I20260811 04:41:57.665373 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.655549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.22.144.190:38589)
0811 04:41:57.655702 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.655707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.655908 (+ 201us) server_negotiation.cc:407] Connection header received
0811 04:41:57.656065 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.656068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.656112 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.656183 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.657023 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.657911 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.659030 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.659254 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.659361 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.661956 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.661960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.661963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.661990 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.663437 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.663906 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.663909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.663911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.664005 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.664292 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.664294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.664295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.664474 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.664560 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.665098 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.665239 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 04:41:57.669435 31384 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:39840:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 04:41:57.670269 31384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:57.675596 31535 tablet_service.cc:1467] Processing CreateTablet for tablet 31bbad94dddd4d2ca4b539dcb7a4c7aa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca98aa62ae384a6aa49a7f855219df2b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:57.675670 31849 tablet_service.cc:1467] Processing CreateTablet for tablet 31bbad94dddd4d2ca4b539dcb7a4c7aa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca98aa62ae384a6aa49a7f855219df2b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:57.675632 31692 tablet_service.cc:1467] Processing CreateTablet for tablet 31bbad94dddd4d2ca4b539dcb7a4c7aa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca98aa62ae384a6aa49a7f855219df2b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 04:41:57.676026 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bbad94dddd4d2ca4b539dcb7a4c7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.676025 31692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bbad94dddd4d2ca4b539dcb7a4c7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.676025 31849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31bbad94dddd4d2ca4b539dcb7a4c7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:57.678581 31960 tablet_bootstrap.cc:492] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Bootstrap starting.
I20260811 04:41:57.678941 31958 tablet_bootstrap.cc:492] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: Bootstrap starting.
I20260811 04:41:57.679522 31960 tablet_bootstrap.cc:654] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.680207 31959 tablet_bootstrap.cc:492] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: Bootstrap starting.
I20260811 04:41:57.680207 31958 tablet_bootstrap.cc:654] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.680806 31960 tablet_bootstrap.cc:492] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: No bootstrap required, opened a new log
I20260811 04:41:57.680898 31960 ts_tablet_manager.cc:1397] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:57.681200 31959 tablet_bootstrap.cc:654] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:57.681363 31960 raft_consensus.cc:348] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.681535 31960 raft_consensus.cc:374] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.681588 31960 raft_consensus.cc:729] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.681739 31960 consensus_queue.cc:260] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.682044 31960 ts_tablet_manager.cc:1428] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:57.682161 31958 tablet_bootstrap.cc:492] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: No bootstrap required, opened a new log
I20260811 04:41:57.682260 31958 ts_tablet_manager.cc:1397] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260811 04:41:57.682524 31959 tablet_bootstrap.cc:492] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: No bootstrap required, opened a new log
I20260811 04:41:57.682605 31959 ts_tablet_manager.cc:1397] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 04:41:57.682674 31958 raft_consensus.cc:348] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.682808 31958 raft_consensus.cc:374] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.682847 31958 raft_consensus.cc:729] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.683070 31981 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.683110 31959 raft_consensus.cc:348] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.683197 31959 raft_consensus.cc:374] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:57.683218 31959 raft_consensus.cc:729] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7b935433e1456a87188e5cbe0c588d, State: Initialized, Role: FOLLOWER
I20260811 04:41:57.683211 31981 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.683344 31959 consensus_queue.cc:260] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.683583 31966 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.683733 31966 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.683498 31958 consensus_queue.cc:260] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.684003 31959 ts_tablet_manager.cc:1428] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:57.684188 31968 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.684343 31968 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.684970 31958 ts_tablet_manager.cc:1428] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:57.795994 31966 raft_consensus.cc:482] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:57.796171 31966 raft_consensus.cc:504] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.797613 31966 leader_election.cc:288] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725)
I20260811 04:41:57.802069 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.797903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42365 (local address 127.22.144.130:33607)
0811 04:41:57.798324 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.798336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.798438 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.798720 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.798724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.798747 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.798888 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.798897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.799799 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.799803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.800818 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.800830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.801622 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.801626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.801730 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.801759 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.801843 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.801913 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":66,"thread_start_us":120,"threads_started":1}
I20260811 04:41:57.802112 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.798035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:33607 (local address 127.22.144.129:42365)
0811 04:41:57.798237 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.798243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.798416 (+ 173us) server_negotiation.cc:407] Connection header received
0811 04:41:57.798532 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.798537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.798599 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.798687 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.799007 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.799677 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.800940 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.801468 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.801619 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.801674 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.801884 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.801985 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 04:41:57.802637 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.798108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:38185 (local address 127.22.144.131:45725)
0811 04:41:57.798254 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.798259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.798588 (+ 329us) server_negotiation.cc:407] Connection header received
0811 04:41:57.798762 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.798765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.798813 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.798900 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:41:57.799204 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.799785 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.801265 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.801910 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.802034 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.802080 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.802439 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.802522 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 04:41:57.802583 31550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bbad94dddd4d2ca4b539dcb7a4c7aa" candidate_uuid: "3a7b935433e1456a87188e5cbe0c588d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" is_pre_election: true
I20260811 04:41:57.802780 31550 raft_consensus.cc:2454] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a7b935433e1456a87188e5cbe0c588d in term 0.
I20260811 04:41:57.803020 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.797918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:45725 (local address 127.22.144.130:38185)
0811 04:41:57.798481 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.798496 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.798645 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.798934 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.798936 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.798974 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:41:57.799082 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.799090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.799933 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.799937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.801106 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.801117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.802148 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.802153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.802275 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.802305 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.802774 (+ 469us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.802861 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":78,"thread_start_us":91,"threads_started":1}
I20260811 04:41:57.803200 31641 leader_election.cc:302] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7b935433e1456a87188e5cbe0c588d, 4d7686fd146d4aa3a30f8c3fc0a46fb2; no voters:
I20260811 04:41:57.803503 31966 raft_consensus.cc:2781] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:57.803577 31966 raft_consensus.cc:482] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:57.803618 31966 raft_consensus.cc:3037] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.803746 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bbad94dddd4d2ca4b539dcb7a4c7aa" candidate_uuid: "3a7b935433e1456a87188e5cbe0c588d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" is_pre_election: true
I20260811 04:41:57.803898 31875 raft_consensus.cc:2454] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a7b935433e1456a87188e5cbe0c588d in term 0.
I20260811 04:41:57.804950 31966 raft_consensus.cc:504] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.805290 31966 leader_election.cc:288] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [CANDIDATE]: Term 1 election: Requested vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725)
I20260811 04:41:57.805747 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bbad94dddd4d2ca4b539dcb7a4c7aa" candidate_uuid: "3a7b935433e1456a87188e5cbe0c588d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff37bb35f9934c85b6850ee5ae0515c3"
I20260811 04:41:57.805764 31550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31bbad94dddd4d2ca4b539dcb7a4c7aa" candidate_uuid: "3a7b935433e1456a87188e5cbe0c588d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2"
I20260811 04:41:57.805869 31875 raft_consensus.cc:3037] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.805871 31550 raft_consensus.cc:3037] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:57.806993 31550 raft_consensus.cc:2454] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a7b935433e1456a87188e5cbe0c588d in term 1.
I20260811 04:41:57.807000 31875 raft_consensus.cc:2454] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a7b935433e1456a87188e5cbe0c588d in term 1.
I20260811 04:41:57.807325 31638 leader_election.cc:302] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7b935433e1456a87188e5cbe0c588d, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:57.807511 31966 raft_consensus.cc:2781] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:57.807806 31966 raft_consensus.cc:686] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [term 1 LEADER]: Becoming Leader. State: Replica: 3a7b935433e1456a87188e5cbe0c588d, State: Running, Role: LEADER
I20260811 04:41:57.808056 31966 consensus_queue.cc:237] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:57.810544 32010 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: TxnStatusTablet state changed. Reason: New leader 3a7b935433e1456a87188e5cbe0c588d. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.810657 32010 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:57.810986 32013 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 04:41:57.810909 31380 catalog_manager.cc:5495] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130). New cstate: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } health_report { overall_health: UNKNOWN } } }
I20260811 04:41:57.811136 32013 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 04:41:57.846422 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.842189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34121 (local address 127.0.0.1:56334)
0811 04:41:57.842327 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.842345 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.842488 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.842857 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.842860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.842868 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:57.842987 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.842992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.843843 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.843847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.844618 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.844630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.845106 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.845110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.845207 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.845257 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.846114 (+ 857us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.846132 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.846206 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.846298 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 04:41:57.846422 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.842321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56334 (local address 127.22.144.130:34121)
0811 04:41:57.842476 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.842480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.842496 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:41:57.842675 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.842678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.842738 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.842843 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:57.843104 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.843726 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.844755 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.844981 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.845091 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.845423 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.845998 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.846145 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.846215 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.846272 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 04:41:57.850134 31875 raft_consensus.cc:1261] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Refusing update from remote peer 3a7b935433e1456a87188e5cbe0c588d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:57.850288 31550 raft_consensus.cc:1261] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Refusing update from remote peer 3a7b935433e1456a87188e5cbe0c588d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:57.850661 31966 consensus_queue.cc:1035] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:57.850835 32010 consensus_queue.cc:1035] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:57.852763 31981 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: TxnStatusTablet state changed. Reason: New leader 3a7b935433e1456a87188e5cbe0c588d. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.852890 31981 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.853327 31968 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: TxnStatusTablet state changed. Reason: New leader 3a7b935433e1456a87188e5cbe0c588d. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.854869 31976 mvcc.cc:205] Tried to move back new op lower bound from 7317189909907812352 to 7317189909744779264. Current Snapshot: MvccSnapshot[applied={T|T < 7317189909907812352}]
I20260811 04:41:57.855564 32010 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.855708 32010 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:57.853468 31968 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.858045 31968 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.858161 31968 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P ff37bb35f9934c85b6850ee5ae0515c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.860635 32010 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.860786 32010 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 3a7b935433e1456a87188e5cbe0c588d: This TxnStatusTablet replica's current role is: LEADER
I20260811 04:41:57.862510 31981 tablet_replica.cc:439] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7b935433e1456a87188e5cbe0c588d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } } }
I20260811 04:41:57.862643 31981 tablet_replica.cc:442] T 31bbad94dddd4d2ca4b539dcb7a4c7aa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 04:41:57.873649 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.869689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:51931 (local address 127.22.144.130:34121)
0811 04:41:57.869850 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.869857 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.870097 (+ 240us) server_negotiation.cc:407] Connection header received
0811 04:41:57.870218 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.870224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.870284 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.870404 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:57.870649 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.871245 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.872518 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.872759 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.872847 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.873178 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.873250 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.873313 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.873435 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.873530 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:41:57.873690 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.869583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34121 (local address 127.22.144.129:51931)
0811 04:41:57.869977 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.869992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.870107 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.870412 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.870414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.870423 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:57.870541 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.870548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.871353 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.871357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.872393 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.872405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.872876 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.872879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.872976 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.873036 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.873341 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.873347 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.873416 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.873530 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":148,"threads_started":1}
I20260811 04:41:57.881987 31716 raft_consensus.cc:1261] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Refusing update from remote peer 4d7686fd146d4aa3a30f8c3fc0a46fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:57.882016 31875 raft_consensus.cc:1261] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d7686fd146d4aa3a30f8c3fc0a46fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:57.882512 31981 consensus_queue.cc:1035] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:57.882690 31969 consensus_queue.cc:1035] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:57.886080 31975 mvcc.cc:205] Tried to move back new op lower bound from 7317189910039662592 to 7317189908914049024. Current Snapshot: MvccSnapshot[applied={T|T < 7317189910039662592}]
I20260811 04:41:57.907392 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.902394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:35527 (local address 127.22.144.129:42365)
0811 04:41:57.902523 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.902538 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.902808 (+ 270us) server_negotiation.cc:407] Connection header received
0811 04:41:57.902893 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.902896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.902959 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.903035 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:57.903416 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.904240 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.905625 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.905853 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.905942 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.906776 (+ 834us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.906870 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.906949 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.907154 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.907228 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:41:57.907879 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.902306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42365 (local address 127.22.144.130:35527)
0811 04:41:57.902711 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:57.902723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:57.902815 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:57.903151 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:57.903155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:57.903163 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:57.903281 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.903287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.904386 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.904389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.905398 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:57.905409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.906425 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.906430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:57.906529 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.906591 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:57.906993 (+ 402us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:57.907001 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 04:41:57.907601 (+ 600us) client_negotiation.cc:240] Negotiation successful
0811 04:41:57.907717 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
I20260811 04:41:57.941304 31384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:57.941618 31384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:57.943898 31384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Sending DeleteTablet for 3 replicas of tablet 73a7592f1808495badea78d1033a46fa
I20260811 04:41:57.944655 31849 tablet_service.cc:1514] Processing DeleteTablet for tablet 73a7592f1808495badea78d1033a46fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:57 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54224
I20260811 04:41:57.944774 31535 tablet_service.cc:1514] Processing DeleteTablet for tablet 73a7592f1808495badea78d1033a46fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:57 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:34950
I20260811 04:41:57.944785 31692 tablet_service.cc:1514] Processing DeleteTablet for tablet 73a7592f1808495badea78d1033a46fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:57 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:56332
I20260811 04:41:57.944872 31988 tablet_replica.cc:330] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: stopping tablet replica
I20260811 04:41:57.944980 31989 tablet_replica.cc:330] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: stopping tablet replica
I20260811 04:41:57.944981 31990 tablet_replica.cc:330] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: stopping tablet replica
I20260811 04:41:57.945029 31988 raft_consensus.cc:2229] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:57.945091 31989 raft_consensus.cc:2229] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:57.945091 31990 raft_consensus.cc:2229] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:57.945142 31988 raft_consensus.cc:2258] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:57.945199 31989 raft_consensus.cc:2258] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:57.945287 31990 raft_consensus.cc:2258] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:57.946108 31990 ts_tablet_manager.cc:1905] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:57.946110 31988 ts_tablet_manager.cc:1905] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:57.949472 31988 ts_tablet_manager.cc:1918] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:57.949640 31988 log.cc:1196] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/73a7592f1808495badea78d1033a46fa
I20260811 04:41:57.949472 31990 ts_tablet_manager.cc:1918] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:57.949975 31988 ts_tablet_manager.cc:1939] T 73a7592f1808495badea78d1033a46fa P ff37bb35f9934c85b6850ee5ae0515c3: Deleting consensus metadata
I20260811 04:41:57.950050 31990 log.cc:1196] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/73a7592f1808495badea78d1033a46fa
I20260811 04:41:57.950342 31990 ts_tablet_manager.cc:1939] T 73a7592f1808495badea78d1033a46fa P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting consensus metadata
I20260811 04:41:57.951031 31367 catalog_manager.cc:4841] TS ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725): tablet 73a7592f1808495badea78d1033a46fa (table test-table [id=7e304348d2c9480abe4a0a159b7f7414]) successfully deleted
I20260811 04:41:57.951099 31370 catalog_manager.cc:4841] TS 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365): tablet 73a7592f1808495badea78d1033a46fa (table test-table [id=7e304348d2c9480abe4a0a159b7f7414]) successfully deleted
I20260811 04:41:57.951867 31989 ts_tablet_manager.cc:1905] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:57.954888 31989 ts_tablet_manager.cc:1918] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:57.955025 31989 log.cc:1196] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/73a7592f1808495badea78d1033a46fa
I20260811 04:41:57.955410 31989 ts_tablet_manager.cc:1939] T 73a7592f1808495badea78d1033a46fa P 3a7b935433e1456a87188e5cbe0c588d: Deleting consensus metadata
I20260811 04:41:57.957683 31369 catalog_manager.cc:4841] TS 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121): tablet 73a7592f1808495badea78d1033a46fa (table test-table [id=7e304348d2c9480abe4a0a159b7f7414]) successfully deleted
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 04:41:57.980765 31439 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:41:57.972769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.22.144.190:38589)
0811 04:41:57.972958 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.972974 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.973183 (+ 209us) server_negotiation.cc:407] Connection header received
0811 04:41:57.973341 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.973344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.973393 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.973536 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.974499 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.975127 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.976369 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.976654 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.976760 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.980601 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39850: BlockingRecv error: recv got EOF from 127.0.0.1:39850 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 04:41:57 dist-test-slave-ss4n krb5kdc[31334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM
I20260811 04:41:57.992141 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:57.982697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.22.144.190:38589)
0811 04:41:57.982822 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:57.982827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:57.982863 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:41:57.982996 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:57.982999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:57.983051 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:57.983141 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:41:57.983787 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.984607 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.985414 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:57.985816 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:57.985983 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:57.988589 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:41:57.988593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:41:57.988596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:41:57.988623 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:41:57.990332 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.990758 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.990761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.990763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.990851 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:41:57.991212 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:41:57.991215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:41:57.991217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:41:57.991439 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:41:57.991572 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:57.991833 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 04:41:57.991968 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 04:41:57.995587 31384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:57.996091 31384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:58.001739 31535 tablet_service.cc:1467] Processing CreateTablet for tablet 7f489eb4547d4cfd9ebb53849b4081a1 (DEFAULT_TABLE table=test-table [id=db4e164f24ca49e1b5c77f8796534528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:58.002096 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f489eb4547d4cfd9ebb53849b4081a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:58.002247 31692 tablet_service.cc:1467] Processing CreateTablet for tablet 7f489eb4547d4cfd9ebb53849b4081a1 (DEFAULT_TABLE table=test-table [id=db4e164f24ca49e1b5c77f8796534528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:58.002555 31692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f489eb4547d4cfd9ebb53849b4081a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:58.004248 31960 tablet_bootstrap.cc:492] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Bootstrap starting.
I20260811 04:41:58.004834 31849 tablet_service.cc:1467] Processing CreateTablet for tablet 7f489eb4547d4cfd9ebb53849b4081a1 (DEFAULT_TABLE table=test-table [id=db4e164f24ca49e1b5c77f8796534528]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:58.005138 31849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f489eb4547d4cfd9ebb53849b4081a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:58.005108 31959 tablet_bootstrap.cc:492] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Bootstrap starting.
I20260811 04:41:58.005177 31960 tablet_bootstrap.cc:654] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:58.005929 31959 tablet_bootstrap.cc:654] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:58.007045 31959 tablet_bootstrap.cc:492] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: No bootstrap required, opened a new log
I20260811 04:41:58.007130 31959 ts_tablet_manager.cc:1397] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:58.007256 31960 tablet_bootstrap.cc:492] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: No bootstrap required, opened a new log
I20260811 04:41:58.007329 31960 ts_tablet_manager.cc:1397] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:41:58.007647 31958 tablet_bootstrap.cc:492] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Bootstrap starting.
I20260811 04:41:58.007609 31959 raft_consensus.cc:348] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.007727 31959 raft_consensus.cc:374] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:58.007761 31959 raft_consensus.cc:729] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7b935433e1456a87188e5cbe0c588d, State: Initialized, Role: FOLLOWER
I20260811 04:41:58.007695 31960 raft_consensus.cc:348] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.007807 31960 raft_consensus.cc:374] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:58.007841 31960 raft_consensus.cc:729] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Initialized, Role: FOLLOWER
I20260811 04:41:58.007907 31959 consensus_queue.cc:260] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.008008 31960 consensus_queue.cc:260] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.008234 31959 ts_tablet_manager.cc:1428] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:58.008342 31960 ts_tablet_manager.cc:1428] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:58.008733 31958 tablet_bootstrap.cc:654] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:58.010025 31958 tablet_bootstrap.cc:492] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: No bootstrap required, opened a new log
I20260811 04:41:58.010115 31958 ts_tablet_manager.cc:1397] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 04:41:58.010607 31958 raft_consensus.cc:348] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.010751 31958 raft_consensus.cc:374] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:58.010780 31958 raft_consensus.cc:729] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Initialized, Role: FOLLOWER
I20260811 04:41:58.010900 31958 consensus_queue.cc:260] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.011158 31958 ts_tablet_manager.cc:1428] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 04:41:58.222939 32020 raft_consensus.cc:482] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:58.223110 32020 raft_consensus.cc:504] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.223651 32020 leader_election.cc:288] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:58.224153 31550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f489eb4547d4cfd9ebb53849b4081a1" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" is_pre_election: true
I20260811 04:41:58.224181 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f489eb4547d4cfd9ebb53849b4081a1" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d" is_pre_election: true
I20260811 04:41:58.224375 31716 raft_consensus.cc:2454] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 0.
I20260811 04:41:58.224363 31550 raft_consensus.cc:2454] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 0.
I20260811 04:41:58.224813 31803 leader_election.cc:302] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7b935433e1456a87188e5cbe0c588d, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:58.225020 32020 raft_consensus.cc:2781] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:58.225121 32020 raft_consensus.cc:482] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:58.225175 32020 raft_consensus.cc:3037] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:58.226311 32020 raft_consensus.cc:504] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.226732 32020 leader_election.cc:288] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:58.227144 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f489eb4547d4cfd9ebb53849b4081a1" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d"
I20260811 04:41:58.227171 31550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f489eb4547d4cfd9ebb53849b4081a1" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2"
I20260811 04:41:58.227285 31716 raft_consensus.cc:3037] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:58.227303 31550 raft_consensus.cc:3037] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:58.228214 31716 raft_consensus.cc:2454] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 1.
I20260811 04:41:58.228406 31550 raft_consensus.cc:2454] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 1.
I20260811 04:41:58.228549 31803 leader_election.cc:302] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7b935433e1456a87188e5cbe0c588d, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:58.228756 32020 raft_consensus.cc:2781] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:58.228868 32020 raft_consensus.cc:686] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 LEADER]: Becoming Leader. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Running, Role: LEADER
I20260811 04:41:58.229113 32020 consensus_queue.cc:237] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.231714 31382 catalog_manager.cc:5495] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:58.257665 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:58.253725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.22.144.131:45725)
0811 04:41:58.253854 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:58.253857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:58.253888 (+ 31us) server_negotiation.cc:407] Connection header received
0811 04:41:58.254041 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:58.254044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:58.254104 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:58.254213 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:58.254442 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.254972 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.255934 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.256159 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.256249 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:58.256613 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:58.257125 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:58.257321 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:58.257351 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 04:41:58.257407 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 04:41:58.260536 31716 raft_consensus.cc:1261] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Refusing update from remote peer ff37bb35f9934c85b6850ee5ae0515c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:58.260614 31550 raft_consensus.cc:1261] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Refusing update from remote peer ff37bb35f9934c85b6850ee5ae0515c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:58.261075 32020 consensus_queue.cc:1035] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:58.261253 31968 consensus_queue.cc:1035] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:58.273227 31384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:58.273406 31384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:58.275800 31384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Sending DeleteTablet for 3 replicas of tablet 7f489eb4547d4cfd9ebb53849b4081a1
I20260811 04:41:58.276485 31535 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f489eb4547d4cfd9ebb53849b4081a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:58 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:34950
I20260811 04:41:58.276567 31692 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f489eb4547d4cfd9ebb53849b4081a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:58 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:56332
I20260811 04:41:58.276716 31990 tablet_replica.cc:330] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: stopping tablet replica
I20260811 04:41:58.276772 31989 tablet_replica.cc:330] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: stopping tablet replica
I20260811 04:41:58.276814 31849 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f489eb4547d4cfd9ebb53849b4081a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:58 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54224
I20260811 04:41:58.276862 31990 raft_consensus.cc:2229] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:58.276893 31989 raft_consensus.cc:2229] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:58.276983 31988 tablet_replica.cc:330] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: stopping tablet replica
I20260811 04:41:58.277009 31990 raft_consensus.cc:2258] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:58.277053 31989 raft_consensus.cc:2258] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:58.277083 31988 raft_consensus.cc:2229] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:58.277406 31988 raft_consensus.cc:2258] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:58.277935 31989 ts_tablet_manager.cc:1905] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:58.278662 31988 ts_tablet_manager.cc:1905] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:58.278937 31384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:41:58.279373 31384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:41:58.280655 31989 ts_tablet_manager.cc:1918] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:58.280730 31989 log.cc:1196] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/7f489eb4547d4cfd9ebb53849b4081a1
I20260811 04:41:58.281013 31989 ts_tablet_manager.cc:1939] T 7f489eb4547d4cfd9ebb53849b4081a1 P 3a7b935433e1456a87188e5cbe0c588d: Deleting consensus metadata
I20260811 04:41:58.281085 31988 ts_tablet_manager.cc:1918] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:58.281181 31988 log.cc:1196] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/7f489eb4547d4cfd9ebb53849b4081a1
I20260811 04:41:58.281383 31988 ts_tablet_manager.cc:1939] T 7f489eb4547d4cfd9ebb53849b4081a1 P ff37bb35f9934c85b6850ee5ae0515c3: Deleting consensus metadata
I20260811 04:41:58.281965 31369 catalog_manager.cc:4841] TS 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121): tablet 7f489eb4547d4cfd9ebb53849b4081a1 (table test-table [id=db4e164f24ca49e1b5c77f8796534528]) successfully deleted
I20260811 04:41:58.282629 31990 ts_tablet_manager.cc:1905] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:58.282684 31367 catalog_manager.cc:4841] TS ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725): tablet 7f489eb4547d4cfd9ebb53849b4081a1 (table test-table [id=db4e164f24ca49e1b5c77f8796534528]) successfully deleted
I20260811 04:41:58.285034 31990 ts_tablet_manager.cc:1918] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:41:58.285133 31990 log.cc:1196] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/7f489eb4547d4cfd9ebb53849b4081a1
I20260811 04:41:58.285385 31990 ts_tablet_manager.cc:1939] T 7f489eb4547d4cfd9ebb53849b4081a1 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Deleting consensus metadata
I20260811 04:41:58.286227 31370 catalog_manager.cc:4841] TS 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365): tablet 7f489eb4547d4cfd9ebb53849b4081a1 (table test-table [id=db4e164f24ca49e1b5c77f8796534528]) successfully deleted
I20260811 04:41:58.286216 31692 tablet_service.cc:1467] Processing CreateTablet for tablet 17fabbc843f74f7a9c7966a69db0a6c6 (DEFAULT_TABLE table=test-table [id=8e1d80f60fe24a5599f3966b26b569a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:58.286384 31535 tablet_service.cc:1467] Processing CreateTablet for tablet 17fabbc843f74f7a9c7966a69db0a6c6 (DEFAULT_TABLE table=test-table [id=8e1d80f60fe24a5599f3966b26b569a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:58.286494 31692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fabbc843f74f7a9c7966a69db0a6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:58.286614 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fabbc843f74f7a9c7966a69db0a6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:58.286788 31849 tablet_service.cc:1467] Processing CreateTablet for tablet 17fabbc843f74f7a9c7966a69db0a6c6 (DEFAULT_TABLE table=test-table [id=8e1d80f60fe24a5599f3966b26b569a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:41:58.287056 31849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fabbc843f74f7a9c7966a69db0a6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:41:58.288708 31960 tablet_bootstrap.cc:492] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Bootstrap starting.
I20260811 04:41:58.289036 31959 tablet_bootstrap.cc:492] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Bootstrap starting.
I20260811 04:41:58.289542 31960 tablet_bootstrap.cc:654] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:58.289949 31959 tablet_bootstrap.cc:654] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:58.290108 31958 tablet_bootstrap.cc:492] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Bootstrap starting.
I20260811 04:41:58.291038 31958 tablet_bootstrap.cc:654] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Neither blocks nor log segments found. Creating new log.
I20260811 04:41:58.291306 31959 tablet_bootstrap.cc:492] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: No bootstrap required, opened a new log
I20260811 04:41:58.291404 31959 ts_tablet_manager.cc:1397] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:58.291855 31959 raft_consensus.cc:348] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.291983 31959 raft_consensus.cc:374] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:58.292023 31959 raft_consensus.cc:729] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7b935433e1456a87188e5cbe0c588d, State: Initialized, Role: FOLLOWER
I20260811 04:41:58.292186 31958 tablet_bootstrap.cc:492] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: No bootstrap required, opened a new log
I20260811 04:41:58.292164 31959 consensus_queue.cc:260] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.292261 31958 ts_tablet_manager.cc:1397] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 04:41:58.292261 31960 tablet_bootstrap.cc:492] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: No bootstrap required, opened a new log
I20260811 04:41:58.292338 31960 ts_tablet_manager.cc:1397] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 04:41:58.292443 31959 ts_tablet_manager.cc:1428] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:58.292681 31958 raft_consensus.cc:348] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.292776 31958 raft_consensus.cc:374] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:58.292807 31958 raft_consensus.cc:729] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Initialized, Role: FOLLOWER
I20260811 04:41:58.292752 31960 raft_consensus.cc:348] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.292856 31960 raft_consensus.cc:374] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:41:58.292888 31960 raft_consensus.cc:729] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7686fd146d4aa3a30f8c3fc0a46fb2, State: Initialized, Role: FOLLOWER
I20260811 04:41:58.292917 31958 consensus_queue.cc:260] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.293007 31960 consensus_queue.cc:260] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.293156 31958 ts_tablet_manager.cc:1428] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:41:58.293293 31960 ts_tablet_manager.cc:1428] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 04:41:58.365787 32051 raft_consensus.cc:482] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:41:58.365969 32051 raft_consensus.cc:504] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.366482 32051 leader_election.cc:288] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:58.366930 31556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fabbc843f74f7a9c7966a69db0a6c6" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" is_pre_election: true
I20260811 04:41:58.366928 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fabbc843f74f7a9c7966a69db0a6c6" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d" is_pre_election: true
I20260811 04:41:58.367091 31716 raft_consensus.cc:2454] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 0.
I20260811 04:41:58.367082 31556 raft_consensus.cc:2454] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 0.
I20260811 04:41:58.367429 31803 leader_election.cc:302] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7b935433e1456a87188e5cbe0c588d, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:58.367619 32051 raft_consensus.cc:2781] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:41:58.367690 32051 raft_consensus.cc:482] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:41:58.367753 32051 raft_consensus.cc:3037] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:58.368774 32051 raft_consensus.cc:504] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.369174 32051 leader_election.cc:288] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 election: Requested vote from peers 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365), 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121)
I20260811 04:41:58.369585 31556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fabbc843f74f7a9c7966a69db0a6c6" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2"
I20260811 04:41:58.369585 31716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fabbc843f74f7a9c7966a69db0a6c6" candidate_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7b935433e1456a87188e5cbe0c588d"
I20260811 04:41:58.369724 31556 raft_consensus.cc:3037] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:58.369724 31716 raft_consensus.cc:3037] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:41:58.370929 31716 raft_consensus.cc:2454] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 1.
I20260811 04:41:58.370929 31556 raft_consensus.cc:2454] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff37bb35f9934c85b6850ee5ae0515c3 in term 1.
I20260811 04:41:58.371284 31804 leader_election.cc:302] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d7686fd146d4aa3a30f8c3fc0a46fb2, ff37bb35f9934c85b6850ee5ae0515c3; no voters:
I20260811 04:41:58.371493 32051 raft_consensus.cc:2781] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:41:58.371626 32051 raft_consensus.cc:686] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 LEADER]: Becoming Leader. State: Replica: ff37bb35f9934c85b6850ee5ae0515c3, State: Running, Role: LEADER
I20260811 04:41:58.371786 32051 consensus_queue.cc:237] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } }
I20260811 04:41:58.373845 31384 catalog_manager.cc:5495] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff37bb35f9934c85b6850ee5ae0515c3" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45725 } health_report { overall_health: HEALTHY } } }
I20260811 04:41:58.418433 32062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:58.413344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:45109 (local address 127.22.144.130:34121)
0811 04:41:58.413632 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:58.413636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:58.413696 (+ 60us) server_negotiation.cc:407] Connection header received
0811 04:41:58.413891 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:58.413894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:58.413945 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:58.414025 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:58.414387 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.415119 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.417125 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.417357 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.417461 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:58.417891 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:58.418003 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:58.418083 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:58.418249 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 04:41:58.418307 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":71,"threads_started":1}
I20260811 04:41:58.418488 32061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:58.413229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34121 (local address 127.22.144.131:45109)
0811 04:41:58.413616 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:58.413634 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:58.413760 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:58.414080 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:58.414083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:58.414093 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:58.414244 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:58.414252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.415246 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.415250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:58.416956 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:58.416970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.417545 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.417550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:58.417650 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:58.417712 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:58.418119 (+ 407us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:58.418127 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 04:41:58.418211 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 04:41:58.418302 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":118,"threads_started":1}
I20260811 04:41:58.425180 31716 raft_consensus.cc:1261] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Refusing update from remote peer ff37bb35f9934c85b6850ee5ae0515c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:58.425642 32051 consensus_queue.cc:1035] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7b935433e1456a87188e5cbe0c588d" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:58.425701 31556 raft_consensus.cc:1261] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Refusing update from remote peer ff37bb35f9934c85b6850ee5ae0515c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:41:58.426177 32051 consensus_queue.cc:1035] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d7686fd146d4aa3a30f8c3fc0a46fb2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:41:58.448670 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:58.444621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:60747 (local address 127.22.144.131:45725)
0811 04:41:58.444723 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 04:41:58.444727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:41:58.444740 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:41:58.444826 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:41:58.444829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:41:58.444886 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:41:58.444962 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:41:58.445325 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.445999 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.447318 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.447675 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.447760 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:58.448184 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:41:58.448275 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:41:58.448342 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 04:41:58.448480 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 04:41:58.448540 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 04:41:58.448722 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:58.444299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:45725 (local address 127.22.144.130:60747)
0811 04:41:58.444613 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:58.444630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:41:58.444730 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:41:58.445086 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:41:58.445089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:41:58.445098 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 04:41:58.445204 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:58.445210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.446117 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.446121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:58.447169 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:41:58.447182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:41:58.447884 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:41:58.447888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:41:58.448002 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:41:58.448055 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 04:41:58.448381 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 04:41:58.448390 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:41:58.448468 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 04:41:58.448568 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
I20260811 04:41:58.473644 31384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:58.473822 31384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:41:58.475898 31384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d00de4dd58f249cd996008ba055669e7: Sending DeleteTablet for 3 replicas of tablet 17fabbc843f74f7a9c7966a69db0a6c6
I20260811 04:41:58.476626 31849 tablet_service.cc:1514] Processing DeleteTablet for tablet 17fabbc843f74f7a9c7966a69db0a6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:58 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54224
I20260811 04:41:58.476641 31692 tablet_service.cc:1514] Processing DeleteTablet for tablet 17fabbc843f74f7a9c7966a69db0a6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:58 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:56332
I20260811 04:41:58.476665 31535 tablet_service.cc:1514] Processing DeleteTablet for tablet 17fabbc843f74f7a9c7966a69db0a6c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:41:58 UTC) from {username='oryx', principal='oryx/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:34950
I20260811 04:41:58.476823 31988 tablet_replica.cc:330] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: stopping tablet replica
I20260811 04:41:58.476846 31990 tablet_replica.cc:330] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2: stopping tablet replica
I20260811 04:41:58.476847 31989 tablet_replica.cc:330] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: stopping tablet replica
I20260811 04:41:58.476944 31988 raft_consensus.cc:2229] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:41:58.476944 31990 raft_consensus.cc:2229] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:58.476969 31989 raft_consensus.cc:2229] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:41:58.477059 31990 raft_consensus.cc:2258] T 17fabbc843f74f7a9c7966a69db0a6c6 P 4d7686fd146d4aa3a30f8c3fc0a46fb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:58.477087 31989 raft_consensus.cc:2258] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:58.477174 31988 raft_consensus.cc:2258] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:41:58.477413 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 31466
I20260811 04:41:58.478106 31989 ts_tablet_manager.cc:1905] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:58.480301 31988 ts_tablet_manager.cc:1905] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:41:58.481292 31989 ts_tablet_manager.cc:1918] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:58.481371 31989 log.cc:1196] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/17fabbc843f74f7a9c7966a69db0a6c6
I20260811 04:41:58.481662 31989 ts_tablet_manager.cc:1939] T 17fabbc843f74f7a9c7966a69db0a6c6 P 3a7b935433e1456a87188e5cbe0c588d: Deleting consensus metadata
I20260811 04:41:58.482384 31988 ts_tablet_manager.cc:1918] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 04:41:58.482470 31988 log.cc:1196] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/17fabbc843f74f7a9c7966a69db0a6c6
I20260811 04:41:58.482550 31369 catalog_manager.cc:4841] TS 3a7b935433e1456a87188e5cbe0c588d (127.22.144.130:34121): tablet 17fabbc843f74f7a9c7966a69db0a6c6 (table test-table [id=8e1d80f60fe24a5599f3966b26b569a2]) successfully deleted
I20260811 04:41:58.482749 31988 ts_tablet_manager.cc:1939] T 17fabbc843f74f7a9c7966a69db0a6c6 P ff37bb35f9934c85b6850ee5ae0515c3: Deleting consensus metadata
I20260811 04:41:58.483336 31367 catalog_manager.cc:4841] TS ff37bb35f9934c85b6850ee5ae0515c3 (127.22.144.131:45725): tablet 17fabbc843f74f7a9c7966a69db0a6c6 (table test-table [id=8e1d80f60fe24a5599f3966b26b569a2]) successfully deleted
W20260811 04:41:58.490257 31370 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:42365 (error 108)
I20260811 04:41:58.491421 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:41:58.490986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42365 (local address 127.0.0.1:34964)
0811 04:41:58.491218 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 04:41:58.491289 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:42365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
I20260811 04:41:58.491494 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 31620
W20260811 04:41:58.495689 31370 catalog_manager.cc:4584] TS 4d7686fd146d4aa3a30f8c3fc0a46fb2 (127.22.144.129:42365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17fabbc843f74f7a9c7966a69db0a6c6: Network error: Client connection negotiation failed: client connection to 127.22.144.129:42365: connect: Connection refused (error 111)
I20260811 04:41:58.507134 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 31780
I20260811 04:41:58.516780 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 31350
2026-08-11T04:41:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3573 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:41:58 dist-test-slave-ss4n krb5kdc[32073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:41:58 dist-test-slave-ss4n krb5kdc[32073](info): set up 2 sockets
Aug 11 04:41:58 dist-test-slave-ss4n krb5kdc[32073](info): commencing operation
krb5kdc: starting...
W20260811 04:41:59.680630 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:41:59 dist-test-slave-ss4n krb5kdc[32073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:41:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:41:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:41:59.820703 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40313
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:40313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:41:59.936283 32087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:41:59.936585 32087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:41:59.936628 32087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:41:59.940681 32087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:41:59.940749 32087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:41:59.940770 32087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:41:59.940788 32087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:41:59.940804 32087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:41:59.946193 32087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:40313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:41:59.947234 32087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:41:59.948232 32087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:41:59.956840 32095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:59.956761 32093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:41:59.956761 32092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:41:59.957300 32087 server_base.cc:1034] running on GCE node
I20260811 04:41:59.957899 32087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:41:59.959060 32087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:41:59.960253 32087 hybrid_clock.cc:648] HybridClock initialized: now 1786423319960241 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T04:41:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1447 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:42:00 dist-test-slave-ss4n krb5kdc[32103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:42:00 dist-test-slave-ss4n krb5kdc[32103](info): set up 2 sockets
Aug 11 04:42:00 dist-test-slave-ss4n krb5kdc[32103](info): commencing operation
krb5kdc: starting...
W20260811 04:42:01.146634 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:42:01 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:42:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:42:01.286197 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40435
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:40435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:01.403204 32117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:01.403555 32117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:01.403652 32117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:01.407796 32117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:42:01.407903 32117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:01.407938 32117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:01.407976 32117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:42:01.408013 32117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:42:01.413671 32117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:40435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:01.414824 32117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:01.415897 32117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:01.425057 32126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:01.425062 32123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:01.425086 32122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:01.425482 32117 server_base.cc:1034] running on GCE node
I20260811 04:42:01.426139 32117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:01.427347 32117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:01.428571 32117 hybrid_clock.cc:648] HybridClock initialized: now 1786423321428547 us; error 35 us; skew 500 ppm
Aug 11 04:42:01 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:01.431826 32117 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:42:01.432843 32117 webserver.cc:460] Webserver started at http://127.22.144.190:34261/ using document root <none> and password file <none>
I20260811 04:42:01.433461 32117 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:01.433627 32117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:01.433871 32117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:01.436164 32117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "3beaff93623945f1b5dd3297d3fb8a08"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "dfba1fbce9c6e43f2f475d4dfee78336"
server_key_iv: "7a8bb1d0b4ea3ded35f747c591ac04ae"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.436836 32117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "3beaff93623945f1b5dd3297d3fb8a08"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "dfba1fbce9c6e43f2f475d4dfee78336"
server_key_iv: "7a8bb1d0b4ea3ded35f747c591ac04ae"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.441310 32117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 04:42:01.444309 32133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:01.445838 32117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:01.445986 32117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "3beaff93623945f1b5dd3297d3fb8a08"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "dfba1fbce9c6e43f2f475d4dfee78336"
server_key_iv: "7a8bb1d0b4ea3ded35f747c591ac04ae"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.446115 32117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:01.461964 32117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:01.465564 32117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:01.465785 32117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:01.473222 32117 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:40435
I20260811 04:42:01.473320 32190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:40435 every 8 connection(s)
I20260811 04:42:01.474308 32117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:42:01.477425 32191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:01.483743 32191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Bootstrap starting.
I20260811 04:42:01.484231 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 32117
I20260811 04:42:01.484431 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:42:01.484745 23106 external_mini_cluster.cc:1428] Setting key f5903596c3ecce15056d7767d4cda91c
I20260811 04:42:01.486750 32191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:01.487604 32191 log.cc:824] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:01.489709 32191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: No bootstrap required, opened a new log
I20260811 04:42:01.493605 32191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:01.493878 32191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:01.493961 32191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3beaff93623945f1b5dd3297d3fb8a08, State: Initialized, Role: FOLLOWER
Aug 11 04:42:01 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:42:01.494501 32191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:01.494719 32191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:42:01.494803 32191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:42:01.494920 32191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:01.496093 32191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:01.496467 32191 leader_election.cc:302] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3beaff93623945f1b5dd3297d3fb8a08; no voters:
I20260811 04:42:01.496841 32191 leader_election.cc:288] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:42:01.496982 32197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:42:01.497231 32197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 LEADER]: Becoming Leader. State: Replica: 3beaff93623945f1b5dd3297d3fb8a08, State: Running, Role: LEADER
I20260811 04:42:01.497644 32197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:01.498440 32191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:42:01.499856 32199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3beaff93623945f1b5dd3297d3fb8a08. Latest consensus state: current_term: 1 leader_uuid: "3beaff93623945f1b5dd3297d3fb8a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } } }
I20260811 04:42:01.499861 32198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3beaff93623945f1b5dd3297d3fb8a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } } }
I20260811 04:42:01.500020 32199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:01.500021 32198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:01.500586 32203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:42:01.502485 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:01.486326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59532 (local address 127.22.144.190:40435)
0811 04:42:01.486808 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:01.486827 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:01.486892 (+ 65us) server_negotiation.cc:407] Connection header received
0811 04:42:01.487604 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:01.487622 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:01.488022 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:01.488396 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:01.489288 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:01.490454 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:01.491381 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:01.491891 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:01.492082 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:01.495061 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:01.495073 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:01.495091 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:01.495122 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:01.497750 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:01.498394 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:01.498400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:01.498407 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:01.498567 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:01.498946 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:01.498950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:01.498952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:01.499608 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:01.499848 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:01.500979 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0811 04:42:01.501342 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":167,"threads_started":1}
I20260811 04:42:01.504453 32203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:42:01.511109 32203 catalog_manager.cc:1347] Generated new cluster ID: 4566097f762d454d9c1e678770fd9a9d
I20260811 04:42:01.511214 32203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:42:01.518641 32203 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:42:01.519763 32203 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:42:01.524761 32203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Generated new TSK 0
I20260811 04:42:01.525581 32203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:42:01.565812 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:40435
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:01.701620 32221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:01.701928 32221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:01.702004 32221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:01.706647 32221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:01.706722 32221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:01.706816 32221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:42:01.712853 32221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:01.714066 32221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:01.715140 32221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:01.725332 32229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:01.725338 32227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:01.725332 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:01.725732 32221 server_base.cc:1034] running on GCE node
I20260811 04:42:01.726239 32221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:01.726967 32221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:01.728147 32221 hybrid_clock.cc:648] HybridClock initialized: now 1786423321728126 us; error 36 us; skew 500 ppm
Aug 11 04:42:01 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:01.731091 32221 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:42:01.731976 32221 webserver.cc:460] Webserver started at http://127.22.144.129:35951/ using document root <none> and password file <none>
I20260811 04:42:01.732517 32221 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:01.732578 32221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:01.732748 32221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:01.734838 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "c69dc68e8897442abc4a87f9b4b1f488"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "4bb2afd9d4e8417257a4584ee1fb5b51"
server_key_iv: "2a3d82f6cb1269772dd1e4cf43103da9"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.735384 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "c69dc68e8897442abc4a87f9b4b1f488"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "4bb2afd9d4e8417257a4584ee1fb5b51"
server_key_iv: "2a3d82f6cb1269772dd1e4cf43103da9"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.739289 32221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 04:42:01.742271 32237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:01.743639 32221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:01.743794 32221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "c69dc68e8897442abc4a87f9b4b1f488"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "4bb2afd9d4e8417257a4584ee1fb5b51"
server_key_iv: "2a3d82f6cb1269772dd1e4cf43103da9"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.743935 32221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:01.776690 32221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:01.780272 32221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:01.780498 32221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:01.781157 32221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:01.782102 32221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:01.782182 32221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:01.782256 32221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:01.782305 32221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:01.793008 32221 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:41697
I20260811 04:42:01.793046 32367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:41697 every 8 connection(s)
I20260811 04:42:01.794101 32221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:42:01.795192 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 32221
I20260811 04:42:01.795348 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:42:01.795704 23106 external_mini_cluster.cc:1428] Setting key 619885f3fec26b587d8e7264cbd1717b
Aug 11 04:42:01 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:42:01.810571 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:01.796370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:52603 (local address 127.22.144.190:40435)
0811 04:42:01.796534 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:01.796539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:01.797823 (+ 1284us) server_negotiation.cc:407] Connection header received
0811 04:42:01.798863 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:01.798869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:01.798933 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:01.799042 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:01.800835 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:01.801438 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:01.802756 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:01.803015 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:01.803130 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:01.806191 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:01.806194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:01.806196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:01.806224 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:01.808008 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:01.808717 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:01.808726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:01.808727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:01.808813 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:01.809204 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:01.809209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:01.809211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:01.809436 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:01.809606 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:01.810213 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 04:42:01.810360 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 04:42:01.811444 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:01.796814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:52603)
0811 04:42:01.797641 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:01.797700 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:01.798649 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:01.799253 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:01.799260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:01.799833 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:01.800656 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:01.800682 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:01.801657 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:01.801660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:01.802623 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:01.802633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:01.803179 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:01.803183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:01.803353 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:01.804092 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:01.804110 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:01.806002 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:01.808145 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:01.808152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:01.808171 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:01.808571 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:01.808916 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:01.808921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:01.808923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:01.809095 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:01.809618 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:01.809628 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:01.809918 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 04:42:01.810140 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 04:42:01.810514 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":163,"threads_started":1}
I20260811 04:42:01.812907 32368 heartbeater.cc:347] Connected to a master server at 127.22.144.190:40435
I20260811 04:42:01.813192 32368 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:01.813781 32368 heartbeater.cc:511] Master 127.22.144.190:40435 requested a full tablet report, sending...
I20260811 04:42:01.815411 32151 ts_manager.cc:194] Registered new tserver with Master: c69dc68e8897442abc4a87f9b4b1f488 (127.22.144.129:41697)
I20260811 04:42:01.816656 32151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:52603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:42:01.830065 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:40435
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:01.949878 32374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:01.950178 32374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:01.950263 32374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:01.954464 32374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:01.954537 32374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:01.954634 32374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:42:01.960539 32374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:01.961778 32374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:01.962898 32374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:01.973657 32384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:01.973718 32374 server_base.cc:1034] running on GCE node
W20260811 04:42:01.973644 32379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:01.973644 32380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:01.974380 32374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:01.975118 32374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:01.976322 32374 hybrid_clock.cc:648] HybridClock initialized: now 1786423321976309 us; error 34 us; skew 500 ppm
Aug 11 04:42:01 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:01.983094 32374 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:42:01.984105 32374 webserver.cc:460] Webserver started at http://127.22.144.130:33645/ using document root <none> and password file <none>
I20260811 04:42:01.984674 32374 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:01.984740 32374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:01.984920 32374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:01.987203 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "aaebbc117ba04acf897f93d08a6796e3"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "d11c17292130cdf27973aaabe82cc706"
server_key_iv: "a5c65f2613be1c97a3a4e52eebc2e690"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.987739 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "aaebbc117ba04acf897f93d08a6796e3"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "d11c17292130cdf27973aaabe82cc706"
server_key_iv: "a5c65f2613be1c97a3a4e52eebc2e690"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.991932 32374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:42:01.994737 32391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:01.996225 32374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:01.996351 32374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "aaebbc117ba04acf897f93d08a6796e3"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "d11c17292130cdf27973aaabe82cc706"
server_key_iv: "a5c65f2613be1c97a3a4e52eebc2e690"
server_key_version: "encryptionkey@0"
I20260811 04:42:01.996471 32374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:02.032940 32374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:02.036840 32374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:02.037076 32374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:02.037811 32374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:02.038741 32374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:02.038797 32374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.038863 32374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:02.038897 32374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.049394 32374 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:46591
I20260811 04:42:02.049471 32521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:46591 every 8 connection(s)
I20260811 04:42:02.050572 32374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:42:02.058909 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 32374
I20260811 04:42:02.059083 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:42:02.059427 23106 external_mini_cluster.cc:1428] Setting key fb363d030b1ae7d853598081c206ed2c
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423321, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:42:02.066190 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.052597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:36991 (local address 127.22.144.190:40435)
0811 04:42:02.052771 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:02.052775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:02.054045 (+ 1270us) server_negotiation.cc:407] Connection header received
0811 04:42:02.055059 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:02.055062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:02.055126 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:02.055220 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:02.056702 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.057242 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.058620 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.058884 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.058990 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.062289 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:02.062292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:02.062294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:02.062319 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:02.063895 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:02.064468 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:02.064471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:02.064473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:02.064555 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:02.064923 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:02.064926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:02.064928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:02.065135 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:02.065240 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:02.065865 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 04:42:02.066001 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 04:42:02.067067 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.053061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:36991)
0811 04:42:02.053883 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:02.053927 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:02.054743 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:02.055441 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:02.055449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:02.055961 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:02.056506 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.056522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.057381 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.057384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.058455 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.058467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.059074 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.059078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.059304 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.060205 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:02.060226 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:02.062091 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:02.064028 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:02.064036 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:02.064054 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:02.064346 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:02.064656 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:02.064659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:02.064661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:02.064817 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:02.065255 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:02.065264 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:02.065604 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 04:42:02.065805 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 04:42:02.066172 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":117,"threads_started":1}
I20260811 04:42:02.068449 32523 heartbeater.cc:347] Connected to a master server at 127.22.144.190:40435
I20260811 04:42:02.068770 32523 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:02.069523 32523 heartbeater.cc:511] Master 127.22.144.190:40435 requested a full tablet report, sending...
I20260811 04:42:02.070911 32151 ts_manager.cc:194] Registered new tserver with Master: aaebbc117ba04acf897f93d08a6796e3 (127.22.144.130:46591)
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:42:02.071782 32151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:36991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:42:02.094250 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:40435
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:02.213709 32528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:02.214005 32528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:02.214051 32528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:02.218261 32528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:02.218338 32528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:02.218400 32528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:42:02.225160 32528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:02.226362 32528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:02.227448 32528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:02.237874 32533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:02.237927 32534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:02.237912 32528 server_base.cc:1034] running on GCE node
W20260811 04:42:02.237867 32536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:02.238545 32528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:02.239252 32528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:02.240465 32528 hybrid_clock.cc:648] HybridClock initialized: now 1786423322240444 us; error 34 us; skew 500 ppm
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423322, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:02.243530 32528 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:42:02.244491 32528 webserver.cc:460] Webserver started at http://127.22.144.131:45841/ using document root <none> and password file <none>
I20260811 04:42:02.245076 32528 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:02.245170 32528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:02.245410 32528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:02.247743 32528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "7d78e37996a848df90d83d829579f346"
format_stamp: "Formatted at 2026-08-11 04:42:02 on dist-test-slave-ss4n"
server_key: "8b657dbd06c2886756579c4bf4a7c05e"
server_key_iv: "234e1ba874c2e026a5cbd54e75c52b56"
server_key_version: "encryptionkey@0"
I20260811 04:42:02.248358 32528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "7d78e37996a848df90d83d829579f346"
format_stamp: "Formatted at 2026-08-11 04:42:02 on dist-test-slave-ss4n"
server_key: "8b657dbd06c2886756579c4bf4a7c05e"
server_key_iv: "234e1ba874c2e026a5cbd54e75c52b56"
server_key_version: "encryptionkey@0"
I20260811 04:42:02.252427 32528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:42:02.255218 32544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.256445 32528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 04:42:02.256577 32528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "7d78e37996a848df90d83d829579f346"
format_stamp: "Formatted at 2026-08-11 04:42:02 on dist-test-slave-ss4n"
server_key: "8b657dbd06c2886756579c4bf4a7c05e"
server_key_iv: "234e1ba874c2e026a5cbd54e75c52b56"
server_key_version: "encryptionkey@0"
I20260811 04:42:02.256719 32528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:02.282976 32528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:02.286470 32528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:02.286692 32528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:02.287324 32528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:02.288182 32528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:02.288250 32528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.288318 32528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:02.288364 32528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.297853 32528 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:37453
I20260811 04:42:02.297888 32673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:37453 every 8 connection(s)
I20260811 04:42:02.298836 32528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:42:02.302922 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 32528
I20260811 04:42:02.303081 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:42:02.303387 23106 external_mini_cluster.cc:1428] Setting key a14f57972ce8a24d7c7db661de8dea74
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423322, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:42:02.314199 32195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.300822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:53905 (local address 127.22.144.190:40435)
0811 04:42:02.300978 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:02.300983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:02.302297 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 04:42:02.303516 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:02.303525 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:02.303644 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:02.303745 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:02.305207 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.306048 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.307314 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.307572 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.307670 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.310241 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:02.310244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:02.310251 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:02.310280 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:02.311978 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:02.312544 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:02.312547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:02.312548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:02.312653 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:02.312982 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:02.312985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:02.312986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:02.313188 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:02.313275 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:02.313935 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 04:42:02.314062 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 04:42:02.314992 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.301301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:53905)
0811 04:42:02.302099 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:02.302153 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:02.303210 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:02.303925 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:02.303933 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:02.304395 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:02.305010 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.305026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.306187 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.306189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.307179 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.307190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.307721 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.307724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.307873 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.308505 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:02.308529 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:02.310051 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:02.312118 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:02.312125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:02.312146 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:02.312410 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:02.312752 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:02.312754 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:02.312756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:02.312896 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:02.313278 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:02.313285 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:02.313652 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 04:42:02.313826 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 04:42:02.314116 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":161,"threads_started":1}
I20260811 04:42:02.316282 32674 heartbeater.cc:347] Connected to a master server at 127.22.144.190:40435
I20260811 04:42:02.316584 32674 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:02.317195 32674 heartbeater.cc:511] Master 127.22.144.190:40435 requested a full tablet report, sending...
I20260811 04:42:02.318557 32151 ts_manager.cc:194] Registered new tserver with Master: 7d78e37996a848df90d83d829579f346 (127.22.144.131:37453)
I20260811 04:42:02.319233 32151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:53905
I20260811 04:42:02.330385 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab.
Entry for principal oryx/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab.
WARNING: no policy specified for oryx/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab.
Entry for principal oryx/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab.
WARNING: no policy specified for oryx/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab.
Entry for principal oryx/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab.
I20260811 04:42:02.420626 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 32221
I20260811 04:42:02.430455 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 32374
I20260811 04:42:02.438297 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 32528
I20260811 04:42:02.446167 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 32117
I20260811 04:42:02.457119 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40435
--webserver_interface=127.22.144.190
--webserver_port=34261
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:40435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:02.576493 32686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:02.576807 32686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:02.576854 32686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:02.581050 32686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:42:02.581130 32686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:02.581148 32686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:02.581166 32686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:42:02.581182 32686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:42:02.586760 32686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:40435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:40435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:02.587940 32686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:02.589013 32686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:02.598479 32694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:02.598467 32692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:02.598467 32696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:02.598996 32686 server_base.cc:1034] running on GCE node
I20260811 04:42:02.599581 32686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:02.600793 32686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:02.602005 32686 hybrid_clock.cc:648] HybridClock initialized: now 1786423322601991 us; error 45 us; skew 500 ppm
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423322, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:02.605681 32686 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:42:02.606822 32686 webserver.cc:460] Webserver started at http://127.22.144.190:34261/ using document root <none> and password file <none>
I20260811 04:42:02.607482 32686 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:02.607613 32686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:02.612202 32686 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:42:02.614957 32704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.616437 32686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:42:02.616637 32686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "3beaff93623945f1b5dd3297d3fb8a08"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "dfba1fbce9c6e43f2f475d4dfee78336"
server_key_iv: "7a8bb1d0b4ea3ded35f747c591ac04ae"
server_key_version: "encryptionkey@0"
I20260811 04:42:02.617244 32686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:02.639178 32686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:02.642824 32686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:02.643052 32686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:02.651245 32686 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:40435
I20260811 04:42:02.651266 32766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:40435 every 8 connection(s)
I20260811 04:42:02.652421 32686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:42:02.654937 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 32686
I20260811 04:42:02.655416 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:41697
--local_ip_for_outbound_sockets=127.22.144.129
--tserver_master_addrs=127.22.144.190:40435
--webserver_port=35951
--webserver_interface=127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:42:02.657126 300 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:42:02.658958 300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Bootstrap starting.
I20260811 04:42:02.678100 300 log.cc:824] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:02.685614 300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:42:02.686043 300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Bootstrap complete.
I20260811 04:42:02.691547 300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:02.692020 300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3beaff93623945f1b5dd3297d3fb8a08, State: Initialized, Role: FOLLOWER
I20260811 04:42:02.692634 300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:02.692790 300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:42:02.692885 300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:42:02.693012 300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 1 FOLLOWER]: Advancing to term 2
I20260811 04:42:02.695343 300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:02.695715 300 leader_election.cc:302] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3beaff93623945f1b5dd3297d3fb8a08; no voters:
I20260811 04:42:02.696105 300 leader_election.cc:288] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 04:42:02.696206 304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 04:42:02.696434 304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [term 2 LEADER]: Becoming Leader. State: Replica: 3beaff93623945f1b5dd3297d3fb8a08, State: Running, Role: LEADER
I20260811 04:42:02.696759 304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } }
I20260811 04:42:02.697535 300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:42:02.697867 304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3beaff93623945f1b5dd3297d3fb8a08. Latest consensus state: current_term: 2 leader_uuid: "3beaff93623945f1b5dd3297d3fb8a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } } }
I20260811 04:42:02.698019 304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:02.698504 305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3beaff93623945f1b5dd3297d3fb8a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3beaff93623945f1b5dd3297d3fb8a08" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 40435 } } }
I20260811 04:42:02.698616 305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:02.703505 320 catalog_manager.cc:1259] Loaded cluster ID: 4566097f762d454d9c1e678770fd9a9d
I20260811 04:42:02.703610 320 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: loading cluster ID for follower catalog manager: success
I20260811 04:42:02.705690 320 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: acquiring CA information for follower catalog manager: success
I20260811 04:42:02.707017 320 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:42:02.707177 308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:42:02.707933 308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:42:02.708403 308 catalog_manager.cc:1259] Loaded cluster ID: 4566097f762d454d9c1e678770fd9a9d
I20260811 04:42:02.708482 308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:42:02.709090 308 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:42:02.709594 308 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3beaff93623945f1b5dd3297d3fb8a08: Loaded TSK: 0
I20260811 04:42:02.710300 308 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 04:42:02.791034 302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:02.791370 302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:02.791451 302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:02.795838 302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:02.795944 302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:02.796042 302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:42:02.802269 302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.129.keytab
--principal=oryx/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:41697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=35951
--tserver_master_addrs=127.22.144.190:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:02.803469 302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:02.804557 302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:02.816849 328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:02.816840 329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:02.816843 333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:02.817246 302 server_base.cc:1034] running on GCE node
I20260811 04:42:02.817736 302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:02.818501 302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:02.819710 302 hybrid_clock.cc:648] HybridClock initialized: now 1786423322819687 us; error 49 us; skew 500 ppm
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423322, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:02.823205 302 init.cc:377] Logged in from keytab as oryx/127.22.144.129@KRBTEST.COM (short username oryx)
I20260811 04:42:02.824189 302 webserver.cc:460] Webserver started at http://127.22.144.129:35951/ using document root <none> and password file <none>
I20260811 04:42:02.824761 302 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:02.824838 302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:02.829218 302 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:42:02.831866 341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.833745 302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:02.833889 302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "c69dc68e8897442abc4a87f9b4b1f488"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "4bb2afd9d4e8417257a4584ee1fb5b51"
server_key_iv: "2a3d82f6cb1269772dd1e4cf43103da9"
server_key_version: "encryptionkey@0"
I20260811 04:42:02.834502 302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:02.850905 302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:02.854713 302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:02.854954 302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:02.855664 302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:02.856637 302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:02.856712 302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.856791 302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:02.856834 302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:02.869204 302 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:41697
I20260811 04:42:02.869223 466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:41697 every 8 connection(s)
I20260811 04:42:02.870458 302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:42:02.873812 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 302
I20260811 04:42:02.874398 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:46591
--local_ip_for_outbound_sockets=127.22.144.130
--tserver_master_addrs=127.22.144.190:40435
--webserver_port=33645
--webserver_interface=127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:02.890408 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.873098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:43847)
0811 04:42:02.876389 (+ 3291us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:02.876441 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:02.877891 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:02.879414 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:02.879422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:02.879903 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:02.880462 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.880482 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.882960 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.882963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.884017 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.884029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.885355 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.885359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.885701 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.886557 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:02.886575 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:02.889535 (+ 2960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3083,"thread_start_us":152,"threads_started":1}
W20260811 04:42:02.890846 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:02.873010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:43847 (local address 127.22.144.190:40435)
0811 04:42:02.873736 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:02.873763 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:02.876615 (+ 2852us) server_negotiation.cc:407] Connection header received
0811 04:42:02.878405 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:02.878427 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:02.878880 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:02.879359 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:02.880727 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.882763 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.884187 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.885141 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.885379 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.889847 (+ 4468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:43847: BlockingRecv error: recv got EOF from 127.22.144.129:43847 (error 108)
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":145,"threads_started":1}
W20260811 04:42:02.890990 467 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:40435 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.144.190:40435: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:02.897809 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.891643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:45211)
0811 04:42:02.891763 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:02.891778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:02.891858 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:02.892132 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:02.892134 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:02.892278 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:02.892377 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.892381 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.893630 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.893632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.894173 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.894180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.894851 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.894855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.894948 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.895567 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:02.895581 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:02.897707 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 04:42:02.898056 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:02.891742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:45211 (local address 127.22.144.190:40435)
0811 04:42:02.891870 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:02.891876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:02.891893 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:42:02.891956 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:02.891960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:02.892016 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:02.892121 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:02.892492 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.893512 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.894287 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.894704 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.894823 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.897908 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:45211: BlockingRecv error: recv got EOF from 127.22.144.129:45211 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:02 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:02.904726 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:02.898427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:38411)
0811 04:42:02.898899 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:02.898920 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:02.899037 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:02.899308 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:02.899310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:02.899467 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:02.899555 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.899560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.900722 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.900725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.901330 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:02.901339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.901986 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.901990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:02.902070 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.902619 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:02.902631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:02.904550 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":137,"threads_started":1}
W20260811 04:42:02.904826 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:02.898494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:38411 (local address 127.22.144.190:40435)
0811 04:42:02.898620 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:02.898625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:02.899017 (+ 392us) server_negotiation.cc:407] Connection header received
0811 04:42:02.899131 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:02.899135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:02.899190 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:02.899297 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:02.899699 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.900548 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.901454 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:02.901843 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:02.901952 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:02.904716 (+ 2764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:38411: BlockingRecv error: recv got EOF from 127.22.144.129:38411 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 04:42:02.905176 467 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 04:42:03.001470 471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:03.001768 471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:03.001811 471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:03.006305 471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:03.006403 471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:03.006546 471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:42:03.011989 471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.130.keytab
--principal=oryx/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:46591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=33645
--tserver_master_addrs=127.22.144.190:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:03.013096 471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:03.014194 471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:03.024488 480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:03.024556 483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:03.024556 479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:03.024750 471 server_base.cc:1034] running on GCE node
I20260811 04:42:03.025241 471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:03.025950 471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:03.027143 471 hybrid_clock.cc:648] HybridClock initialized: now 1786423323027094 us; error 63 us; skew 500 ppm
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423323, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:03.030162 471 init.cc:377] Logged in from keytab as oryx/127.22.144.130@KRBTEST.COM (short username oryx)
I20260811 04:42:03.031168 471 webserver.cc:460] Webserver started at http://127.22.144.130:33645/ using document root <none> and password file <none>
I20260811 04:42:03.031705 471 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:03.031781 471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:03.036012 471 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:42:03.038401 490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:03.039865 471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:42:03.039973 471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "aaebbc117ba04acf897f93d08a6796e3"
format_stamp: "Formatted at 2026-08-11 04:42:01 on dist-test-slave-ss4n"
server_key: "d11c17292130cdf27973aaabe82cc706"
server_key_iv: "a5c65f2613be1c97a3a4e52eebc2e690"
server_key_version: "encryptionkey@0"
I20260811 04:42:03.040584 471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:03.064257 471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:03.068080 471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:03.068313 471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:03.068979 471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:03.069909 471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:03.069983 471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:03.070057 471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:03.070098 471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:03.079577 471 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:46591
I20260811 04:42:03.079674 615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:46591 every 8 connection(s)
I20260811 04:42:03.080704 471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:42:03.083448 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 471
I20260811 04:42:03.084017 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:37453
--local_ip_for_outbound_sockets=127.22.144.131
--tserver_master_addrs=127.22.144.190:40435
--webserver_port=45841
--webserver_interface=127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 04:42:03.092779 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.082886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:54103 (local address 127.22.144.190:40435)
0811 04:42:03.083070 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.083075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.083971 (+ 896us) server_negotiation.cc:407] Connection header received
0811 04:42:03.085424 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.085432 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.085553 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.085686 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.086905 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.087458 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.088647 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.088893 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.089009 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.092648 (+ 3639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:54103: BlockingRecv error: recv got EOF from 127.22.144.130:54103 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 04:42:03.093467 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.083263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:54103)
0811 04:42:03.083789 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.083846 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.085178 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.085862 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.085870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.086229 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.086747 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.086763 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.087602 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.087605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.088514 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.088525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.089059 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.089063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.089264 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.089991 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.090008 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.092504 (+ 2496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":140,"threads_started":1}
W20260811 04:42:03.094080 617 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:40435 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.144.190:40435: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:03.100138 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.094622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:56339)
0811 04:42:03.094855 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.094867 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.094971 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.095228 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.095231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.095402 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.095498 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.095504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.096226 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.096231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.096780 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.096788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.097258 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.097262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.097362 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.098007 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.098024 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.099982 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 04:42:03.100244 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.094724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:56339 (local address 127.22.144.190:40435)
0811 04:42:03.094825 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.094829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.094926 (+ 97us) server_negotiation.cc:407] Connection header received
0811 04:42:03.095077 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.095080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.095128 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.095212 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.095610 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.096115 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.096899 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.097116 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.097230 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.100114 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:56339: BlockingRecv error: recv got EOF from 127.22.144.130:56339 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:03.106314 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.100857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:42515)
0811 04:42:03.100969 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.100983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.101060 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.101311 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.101314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.101474 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.101606 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.101612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.102351 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.102356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.102892 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.102900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.103354 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.103357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.103438 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.104013 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.104024 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.106127 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 04:42:03.106381 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.100976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:42515 (local address 127.22.144.190:40435)
0811 04:42:03.101105 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.101110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.101125 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:42:03.101176 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.101179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.101213 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.101295 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.101722 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.102239 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.103017 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.103239 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.103339 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.106265 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:42515: BlockingRecv error: recv got EOF from 127.22.144.130:42515 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 04:42:03.106662 617 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 04:42:03.207890 621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:03.208187 621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:03.208256 621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:03.212620 621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:03.212687 621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:03.212774 621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:42:03.218329 621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/krb5kdc/oryx_127.22.144.131.keytab
--principal=oryx/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:37453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=45841
--tserver_master_addrs=127.22.144.190:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:03.219442 621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:03.220702 621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:03.232232 628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:03.232285 627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:03.232285 621 server_base.cc:1034] running on GCE node
W20260811 04:42:03.232474 630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:03.233028 621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:03.233783 621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:03.234979 621 hybrid_clock.cc:648] HybridClock initialized: now 1786423323234961 us; error 26 us; skew 500 ppm
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423323, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:03.238018 621 init.cc:377] Logged in from keytab as oryx/127.22.144.131@KRBTEST.COM (short username oryx)
I20260811 04:42:03.239100 621 webserver.cc:460] Webserver started at http://127.22.144.131:45841/ using document root <none> and password file <none>
I20260811 04:42:03.239729 621 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:03.239856 621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:03.244428 621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:42:03.247117 637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:03.248530 621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:42:03.248692 621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "7d78e37996a848df90d83d829579f346"
format_stamp: "Formatted at 2026-08-11 04:42:02 on dist-test-slave-ss4n"
server_key: "8b657dbd06c2886756579c4bf4a7c05e"
server_key_iv: "234e1ba874c2e026a5cbd54e75c52b56"
server_key_version: "encryptionkey@0"
I20260811 04:42:03.249331 621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:03.270183 621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:03.273969 621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:03.274199 621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:03.274868 621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:03.275795 621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:03.275878 621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:03.275959 621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:03.276014 621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:03.286397 621 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:37453
I20260811 04:42:03.286423 763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:37453 every 8 connection(s)
I20260811 04:42:03.287555 621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:42:03.292966 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 621
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 04:42:03.301342 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.289543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:56149 (local address 127.22.144.190:40435)
0811 04:42:03.289722 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.289726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.290682 (+ 956us) server_negotiation.cc:407] Connection header received
0811 04:42:03.292243 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.292247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.292309 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.292435 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.294271 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.295081 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.296408 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.296677 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.296810 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.301190 (+ 4380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:56149: BlockingRecv error: recv got EOF from 127.22.144.131:56149 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:42:03.302074 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.290000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:56149)
0811 04:42:03.290490 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.290552 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.292012 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.292628 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.292636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.293086 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.294068 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.294094 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.295229 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.295234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.296259 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.296271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.296853 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.296858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.297066 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.297881 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.297910 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.301047 (+ 3137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":122,"threads_started":1}
W20260811 04:42:03.302632 764 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:40435 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.144.190:40435: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
I20260811 04:42:03.303697 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.294211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.22.144.190:40435)
0811 04:42:03.294517 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.294523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.294541 (+ 18us) server_negotiation.cc:407] Connection header received
0811 04:42:03.294587 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.294591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.294641 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.294756 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.295598 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.296172 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.297128 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.297374 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.297562 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.299260 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:03.299268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:03.299288 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:03.299323 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:03.301249 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:03.301790 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:03.301796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:03.301802 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:03.301929 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:03.302287 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:03.302293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:03.302295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:03.302770 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:03.302990 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:03.303298 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 04:42:03.303566 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":114,"threads_started":1}
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:03.309127 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.303186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:33121)
0811 04:42:03.303377 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.303394 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.303499 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.303800 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.303803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.303995 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.304104 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.304111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.304960 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.304964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.305666 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.305677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.306195 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.306200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.306312 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.306887 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.306903 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.308929 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260811 04:42:03.309175 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.303253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:33121 (local address 127.22.144.190:40435)
0811 04:42:03.303379 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.303385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.303448 (+ 63us) server_negotiation.cc:407] Connection header received
0811 04:42:03.303612 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.303616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.303683 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.303779 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.304235 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.304810 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.305803 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.306066 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.306196 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.309064 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:33121: BlockingRecv error: recv got EOF from 127.22.144.131:33121 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:03.315336 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.309868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:38261)
0811 04:42:03.310025 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.310039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.310141 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.310446 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.310449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.310630 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.310711 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.310717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.311510 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.311514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.312125 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.312135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.312631 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.312635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.312723 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.313264 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.313277 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.315229 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 04:42:03.315478 469 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.310027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:38261 (local address 127.22.144.190:40435)
0811 04:42:03.310232 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.310237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.310250 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:42:03.310295 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.310297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.310337 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.310427 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.310831 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.311394 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.312257 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.312506 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.312630 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.315356 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:38261: BlockingRecv error: recv got EOF from 127.22.144.131:38261 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 04:42:03.315683 764 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:03 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:03.913148 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:03.906128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:53941)
0811 04:42:03.906484 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:03.906502 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:03.906615 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:03.906996 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:03.906999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:03.907202 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:03.907339 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.907345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.908326 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.908330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.909251 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:03.909262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.910048 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.910055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:03.910149 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.910752 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:03.910765 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:03.912971 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":117,"threads_started":1}
W20260811 04:42:03.913352 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:03.906334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53941 (local address 127.22.144.190:40435)
0811 04:42:03.906660 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:03.906665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:03.906681 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:42:03.906727 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:03.906730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:03.906785 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:03.906887 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:03.907487 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.908153 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.909397 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:03.909905 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:03.910035 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:03.913160 (+ 3125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:53941: BlockingRecv error: recv got EOF from 127.22.144.129:53941 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":99,"threads_started":1}
Aug 11 04:42:04 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:04 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:04.113566 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:04.107343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:52993)
0811 04:42:04.107736 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:04.107754 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:04.107902 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:04.108185 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:04.108188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:04.108402 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:04.108502 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:04.108509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.109401 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.109406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:04.110073 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:04.110083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.110552 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.110556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:04.110643 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:04.111220 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:04.111231 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:04.113360 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":110,"threads_started":1}
W20260811 04:42:04.113708 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:04.107485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:52993 (local address 127.22.144.190:40435)
0811 04:42:04.107713 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:04.107719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:04.107839 (+ 120us) server_negotiation.cc:407] Connection header received
0811 04:42:04.108006 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:04.108010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:04.108064 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:04.108153 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:04.108690 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.109280 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.110199 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.110433 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.110542 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:04.113576 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:52993: BlockingRecv error: recv got EOF from 127.22.144.130:52993 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 11 04:42:04 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:04 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:04.322396 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:04.316462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:48875)
0811 04:42:04.316803 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:04.316816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:04.316958 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:04.317258 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:04.317262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:04.317504 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:04.317611 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:04.317617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.318383 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.318388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:04.319133 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:04.319143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.319599 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.319602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:04.319685 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:04.320236 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:04.320255 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:04.322212 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":94,"threads_started":1}
W20260811 04:42:04.322466 771 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:04.316545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:48875 (local address 127.22.144.190:40435)
0811 04:42:04.316712 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:04.316717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:04.316896 (+ 179us) server_negotiation.cc:407] Connection header received
0811 04:42:04.317081 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:04.317084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:04.317137 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:04.317230 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:04.317736 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.318270 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.319247 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.319474 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.319595 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:04.322347 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:48875: BlockingRecv error: recv got EOF from 127.22.144.131:48875 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 04:42:04 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:04 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:04.920248 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:04.914150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:53075)
0811 04:42:04.914482 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:04.914498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:04.914621 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:04.914919 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:04.914921 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:04.915141 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:04.915248 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:04.915254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.916094 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.916097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:04.916709 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:04.916719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.917243 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.917246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:04.917313 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:04.917956 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:04.917969 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:04.920105 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
W20260811 04:42:04.920418 775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:04.914248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53075 (local address 127.22.144.190:40435)
0811 04:42:04.914521 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:04.914525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:04.914561 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:42:04.914737 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:04.914740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:04.914787 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:04.914871 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:04.915364 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.915961 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.916864 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:04.917126 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:04.917223 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:04.920277 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:53075: BlockingRecv error: recv got EOF from 127.22.144.129:53075 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":63,"threads_started":1}
Aug 11 04:42:05 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:05 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:05.120844 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:05.114448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:57969)
0811 04:42:05.114719 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:05.114732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:05.114842 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:05.115258 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:05.115261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:05.115457 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:05.115671 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:05.115679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.116484 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.116487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:05.117189 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:05.117199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.117799 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.117803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:05.117880 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:05.118588 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:05.118601 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:05.120630 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
W20260811 04:42:05.120901 775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:05.114513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:57969 (local address 127.22.144.190:40435)
0811 04:42:05.114692 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:05.114696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:05.114804 (+ 108us) server_negotiation.cc:407] Connection header received
0811 04:42:05.115035 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:05.115038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:05.115096 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:05.115180 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:05.115813 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.116378 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.117323 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.117661 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.117751 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:05.120806 (+ 3055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:57969: BlockingRecv error: recv got EOF from 127.22.144.130:57969 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 04:42:05 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:05 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:05.329619 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:05.323443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:33955)
0811 04:42:05.323744 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:05.323758 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:05.323850 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:05.324136 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:05.324138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:05.324321 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:05.324422 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:05.324428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.325382 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.325385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:05.326219 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:05.326233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.326861 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.326865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:05.326958 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:05.327624 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:05.327636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:05.329415 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":105,"threads_started":1}
W20260811 04:42:05.329910 775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:05.323646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:33955 (local address 127.22.144.190:40435)
0811 04:42:05.323812 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:05.323816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:05.323833 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:42:05.323942 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:05.323945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:05.323996 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:05.324097 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:05.324644 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.325253 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.326372 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.326725 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.326854 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:05.329626 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:33955: BlockingRecv error: recv got EOF from 127.22.144.131:33955 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 04:42:05 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:05 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:05.927291 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:05.921194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:35261)
0811 04:42:05.921570 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:05.921587 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:05.921727 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:05.922036 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:05.922038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:05.922315 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:05.922441 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:05.922450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.923337 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.923340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:05.923934 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:05.923944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.924372 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.924375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:05.924438 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:05.924910 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:05.924921 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:05.927105 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
W20260811 04:42:05.927352 779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:05.921305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:35261 (local address 127.22.144.190:40435)
0811 04:42:05.921749 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:05.921754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:05.921776 (+ 22us) server_negotiation.cc:407] Connection header received
0811 04:42:05.921827 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:05.921830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:05.921880 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:05.921973 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:05.922623 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.923231 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.924037 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:05.924274 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:05.924354 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:05.927225 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:35261: BlockingRecv error: recv got EOF from 127.22.144.129:35261 (error 108)
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":214,"threads_started":1}
Aug 11 04:42:06 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:06 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:06.127769 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:06.121787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:49069)
0811 04:42:06.122119 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:06.122137 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:06.122260 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:06.122566 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:06.122569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:06.122755 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:06.122862 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:06.122867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.123665 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.123668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:06.124257 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:06.124267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.124764 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.124768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:06.124851 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:06.125414 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:06.125426 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:06.127645 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":126,"threads_started":1}
W20260811 04:42:06.127918 779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:06.121880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:49069 (local address 127.22.144.190:40435)
0811 04:42:06.122047 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:06.122050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:06.122247 (+ 197us) server_negotiation.cc:407] Connection header received
0811 04:42:06.122377 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:06.122380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:06.122430 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:06.122520 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:06.122984 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.123540 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.124377 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.124623 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.124708 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:06.127796 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:49069: BlockingRecv error: recv got EOF from 127.22.144.130:49069 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 04:42:06 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:06 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:06.336562 784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:06.330623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:57951)
0811 04:42:06.331046 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:06.331064 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:06.331184 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:06.331522 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:06.331525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:06.331735 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:06.331859 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:06.331865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.332595 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.332597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:06.333160 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:06.333168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.333805 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.333809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:06.333902 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:06.334495 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:06.334510 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:06.336441 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":157,"threads_started":1}
W20260811 04:42:06.336714 779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:06.330795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:57951 (local address 127.22.144.190:40435)
0811 04:42:06.330946 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:06.330950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:06.331205 (+ 255us) server_negotiation.cc:407] Connection header received
0811 04:42:06.331265 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:06.331267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:06.331321 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:06.331407 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:06.331982 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.332500 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.333294 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.333647 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.333765 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:06.336585 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:57951: BlockingRecv error: recv got EOF from 127.22.144.131:57951 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 04:42:06 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:06 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:06.933732 785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:06.928065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:50263)
0811 04:42:06.928426 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:06.928440 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:06.928546 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:06.928808 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:06.928810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:06.929000 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:06.929105 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:06.929112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.929917 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.929920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:06.930490 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:06.930498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.930906 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.930908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:06.930988 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:06.931621 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:06.931636 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:06.933609 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":75,"threads_started":1}
W20260811 04:42:06.933802 786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:06.928150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:50263 (local address 127.22.144.190:40435)
0811 04:42:06.928437 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:06.928451 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:06.928522 (+ 71us) server_negotiation.cc:407] Connection header received
0811 04:42:06.928640 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:06.928642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:06.928689 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:06.928771 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:06.929220 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.929805 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.930589 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:06.930808 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:06.930886 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:06.933695 (+ 2809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:50263: BlockingRecv error: recv got EOF from 127.22.144.129:50263 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":73,"threads_started":1}
Aug 11 04:42:07 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:07 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:07.138285 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:07.128868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:46601)
0811 04:42:07.129253 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:07.129270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:07.129519 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:07.129833 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:07.129836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:07.130126 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:07.130259 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:07.130267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.131091 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.131094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:07.131799 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:07.131810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.132331 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.132334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:07.132405 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:07.136095 (+ 3690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:07.136108 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:07.138112 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":124,"threads_started":1}
W20260811 04:42:07.138393 786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:07.129049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:46601 (local address 127.22.144.190:40435)
0811 04:42:07.129245 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:07.129250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:07.129364 (+ 114us) server_negotiation.cc:407] Connection header received
0811 04:42:07.129644 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:07.129648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:07.129705 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:07.129812 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:07.130393 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.130974 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.131929 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.132179 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.132283 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:07.138272 (+ 5989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:46601: BlockingRecv error: recv got EOF from 127.22.144.130:46601 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 04:42:07 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:07 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:07.343621 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:07.337378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:57051)
0811 04:42:07.337779 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:07.337798 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:07.337951 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:07.338313 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:07.338316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:07.338546 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:07.338649 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:07.338656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.339487 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.339493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:07.340121 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:07.340130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.340656 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.340660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:07.340764 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:07.341319 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:07.341332 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:07.343443 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":119,"threads_started":1}
W20260811 04:42:07.343798 786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:07.337611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:57051 (local address 127.22.144.190:40435)
0811 04:42:07.337779 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:07.337786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:07.337900 (+ 114us) server_negotiation.cc:407] Connection header received
0811 04:42:07.338110 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:07.338113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:07.338172 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:07.338264 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:07.338783 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.339367 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.340254 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.340524 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.340640 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:07.343655 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:57051: BlockingRecv error: recv got EOF from 127.22.144.131:57051 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 04:42:07 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:07 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:07.940752 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:07.934717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:38485)
0811 04:42:07.935027 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:07.935039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:07.935174 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:07.935476 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:07.935479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:07.935699 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:07.935796 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:07.935803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.936674 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.936677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:07.937248 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:07.937257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.937792 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.937795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:07.937884 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:07.938487 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:07.938504 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:07.940570 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":73,"threads_started":1}
W20260811 04:42:07.940937 791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:07.934847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:38485 (local address 127.22.144.190:40435)
0811 04:42:07.935097 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:07.935100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:07.935118 (+ 18us) server_negotiation.cc:407] Connection header received
0811 04:42:07.935279 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:07.935282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:07.935329 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:07.935450 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:07.935923 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.936485 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.937370 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:07.937671 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:07.937783 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:07.940774 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:38485: BlockingRecv error: recv got EOF from 127.22.144.129:38485 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
Aug 11 04:42:08 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:08 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:08.145885 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:08.139393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:52271)
0811 04:42:08.139785 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:08.139803 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:08.139931 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:08.140267 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:08.140270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:08.140495 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:08.140612 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:08.140617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.141456 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.141458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:08.142118 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:08.142129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.142689 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.142692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:08.142765 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:08.143330 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:08.143341 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:08.145691 (+ 2350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":210,"threads_started":1}
W20260811 04:42:08.145963 791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:08.139605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:52271 (local address 127.22.144.190:40435)
0811 04:42:08.139766 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:08.139771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:08.139897 (+ 126us) server_negotiation.cc:407] Connection header received
0811 04:42:08.140079 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:08.140083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:08.140144 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:08.140227 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:08.140737 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.141318 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.142259 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.142499 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.142651 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:08.145851 (+ 3200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:52271: BlockingRecv error: recv got EOF from 127.22.144.130:52271 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 04:42:08 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:08 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:08.351183 793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:08.344597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:39417)
0811 04:42:08.344955 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:08.344975 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:08.345099 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:08.345395 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:08.345397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:08.345621 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:08.345798 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:08.345805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.346640 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.346643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:08.347256 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:08.347266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.347867 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.347870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:08.347952 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:08.348539 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:08.348555 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:08.350988 (+ 2433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":169,"threads_started":1}
W20260811 04:42:08.351254 791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:08.344700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:39417 (local address 127.22.144.190:40435)
0811 04:42:08.344854 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:08.344858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:08.345070 (+ 212us) server_negotiation.cc:407] Connection header received
0811 04:42:08.345218 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:08.345221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:08.345269 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:08.345381 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:08.345908 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.346513 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.347395 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.347725 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.347823 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:08.351131 (+ 3308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:39417: BlockingRecv error: recv got EOF from 127.22.144.131:39417 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 04:42:08 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:08 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:08.948314 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:08.941772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:45083)
0811 04:42:08.942107 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:08.942123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:08.942278 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:08.942694 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:08.942699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:08.942999 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:08.943146 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:08.943155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.944193 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.944196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:08.944826 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:08.944857 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.945351 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.945354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:08.945427 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:08.946087 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:08.946119 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:08.948195 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":104,"threads_started":1}
W20260811 04:42:08.948479 795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:08.941837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:45083 (local address 127.22.144.190:40435)
0811 04:42:08.942147 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:08.942152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:08.942193 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:42:08.942388 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:08.942391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:08.942469 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:08.942581 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:08.943302 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.944076 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.944982 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:08.945226 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:08.945320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:08.948341 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:45083: BlockingRecv error: recv got EOF from 127.22.144.129:45083 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":87,"threads_started":1}
Aug 11 04:42:09 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:09 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:09.153266 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:09.146787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:43649)
0811 04:42:09.147209 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:09.147222 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:09.147363 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:09.147687 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:09.147690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:09.147928 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:09.148038 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:09.148045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.148893 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.148898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:09.149573 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:09.149582 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.150102 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.150107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:09.150205 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:09.150835 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:09.150848 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:09.153078 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":90,"threads_started":1}
W20260811 04:42:09.153379 795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:09.146916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:43649 (local address 127.22.144.190:40435)
0811 04:42:09.147081 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:09.147087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:09.147310 (+ 223us) server_negotiation.cc:407] Connection header received
0811 04:42:09.147495 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:09.147499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:09.147557 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:09.147657 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:09.148173 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.148764 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.149714 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.149964 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.150081 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:09.153251 (+ 3170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:43649: BlockingRecv error: recv got EOF from 127.22.144.130:43649 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 04:42:09 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:09 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:09.359040 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:09.352254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:57905)
0811 04:42:09.352796 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:09.352810 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:09.352962 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:09.353395 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:09.353397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:09.353706 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:09.353828 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:09.353835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.354753 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.354757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:09.355421 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:09.355432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.355947 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.355950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:09.356023 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:09.356723 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:09.356738 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:09.358866 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":137,"threads_started":1}
W20260811 04:42:09.359143 795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:09.352440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:57905 (local address 127.22.144.190:40435)
0811 04:42:09.352627 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:09.352637 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:09.352904 (+ 267us) server_negotiation.cc:407] Connection header received
0811 04:42:09.353098 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:09.353101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:09.353194 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:09.353282 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:09.353966 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.354583 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.355544 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.355806 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.355895 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:09.359017 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:57905: BlockingRecv error: recv got EOF from 127.22.144.131:57905 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 11 04:42:09 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:09 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:09.955956 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:09.949287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:53345)
0811 04:42:09.949834 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:09.949874 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:09.950031 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:09.950402 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:09.950409 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:09.950680 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:09.950835 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:09.950866 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.951813 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.951816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:09.952409 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:09.952418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.952906 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.952913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:09.953011 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:09.953689 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:09.953703 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:09.955837 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":225,"threads_started":1}
W20260811 04:42:09.956215 798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:09.949399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53345 (local address 127.22.144.190:40435)
0811 04:42:09.949755 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:09.949760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:09.949980 (+ 220us) server_negotiation.cc:407] Connection header received
0811 04:42:09.950163 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:09.950166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:09.950229 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:09.950317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:09.951048 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.951679 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.952517 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:09.952755 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:09.952834 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:09.956071 (+ 3237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:53345: BlockingRecv error: recv got EOF from 127.22.144.129:53345 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":156,"threads_started":1}
Aug 11 04:42:10 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:10 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:10.161005 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:10.154320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:58081)
0811 04:42:10.154781 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:10.154802 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:10.155023 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:10.155402 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:10.155405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:10.155670 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:10.155822 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:10.155833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.156722 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.156729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:10.157472 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:10.157515 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.157986 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.157990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:10.158077 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:10.158692 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:10.158704 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:10.160879 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":93,"threads_started":1}
W20260811 04:42:10.161151 798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:10.154594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:58081 (local address 127.22.144.190:40435)
0811 04:42:10.154783 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:10.154791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:10.154963 (+ 172us) server_negotiation.cc:407] Connection header received
0811 04:42:10.155168 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:10.155171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:10.155246 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:10.155365 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:10.155971 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.156596 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.157630 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.157858 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.157972 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:10.161022 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:58081: BlockingRecv error: recv got EOF from 127.22.144.130:58081 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 04:42:10 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:10 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:10.366761 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:10.360121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:40177)
0811 04:42:10.360471 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:10.360493 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:10.360617 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:10.360944 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:10.360948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:10.361208 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:10.361368 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:10.361378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.362453 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.362458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:10.363215 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:10.363227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.363779 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.363782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:10.363854 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:10.364391 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:10.364404 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:10.366638 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":114,"threads_started":1}
W20260811 04:42:10.366923 798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:10.360260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:40177 (local address 127.22.144.190:40435)
0811 04:42:10.360422 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:10.360427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:10.360612 (+ 185us) server_negotiation.cc:407] Connection header received
0811 04:42:10.360736 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:10.360739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:10.360791 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:10.360947 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:10.361558 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.362316 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.363343 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.363634 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.363721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:10.366798 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:40177: BlockingRecv error: recv got EOF from 127.22.144.131:40177 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 04:42:10 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:10 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:10.963109 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:10.956794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:51939)
0811 04:42:10.957146 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:10.957161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:10.957262 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:10.957547 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:10.957551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:10.957769 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:10.957881 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:10.957887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.958727 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.958731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:10.959387 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:10.959397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.960002 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.960006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:10.960088 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:10.960768 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:10.960782 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:10.962979 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":153,"threads_started":1}
W20260811 04:42:10.963382 803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:10.957002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:51939 (local address 127.22.144.190:40435)
0811 04:42:10.957255 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:10.957258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:10.957272 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:42:10.957339 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:10.957341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:10.957384 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:10.957463 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:10.958002 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.958544 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.959610 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:10.959859 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:10.959946 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:10.963227 (+ 3281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:51939: BlockingRecv error: recv got EOF from 127.22.144.129:51939 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
Aug 11 04:42:11 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:11 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:11.168810 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:11.162026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:43343)
0811 04:42:11.162391 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:11.162416 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:11.162676 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:11.163058 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:11.163061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:11.163305 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:11.163452 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:11.163459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.164517 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.164522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:11.165157 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:11.165167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.165761 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.165765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:11.165877 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:11.166548 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:11.166561 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:11.168619 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":123,"threads_started":1}
W20260811 04:42:11.168876 803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:11.162137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:43343 (local address 127.22.144.190:40435)
0811 04:42:11.162344 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:11.162350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:11.162608 (+ 258us) server_negotiation.cc:407] Connection header received
0811 04:42:11.162836 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:11.162839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:11.162906 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:11.163040 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:11.163603 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.164381 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.165297 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.165628 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.165755 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:11.168758 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:43343: BlockingRecv error: recv got EOF from 127.22.144.130:43343 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 04:42:11 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:11 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:11.373927 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:11.367905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:57415)
0811 04:42:11.368260 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:11.368280 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:11.368426 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:11.368807 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:11.368810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:11.369125 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:11.369280 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:11.369288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.370115 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.370118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:11.370717 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:11.370728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.371169 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.371171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:11.371237 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:11.371745 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:11.371756 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:11.373751 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
W20260811 04:42:11.373994 803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:11.368021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:57415 (local address 127.22.144.190:40435)
0811 04:42:11.368161 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:11.368165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:11.368378 (+ 213us) server_negotiation.cc:407] Connection header received
0811 04:42:11.368543 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:11.368547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:11.368609 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:11.368707 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:11.369404 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.369975 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.370834 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.371068 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.371148 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:11.373882 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:57415: BlockingRecv error: recv got EOF from 127.22.144.131:57415 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 04:42:11 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:11 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:11.970249 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:11.964085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:36539)
0811 04:42:11.964437 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:11.964451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:11.964572 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:11.964873 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:11.964876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:11.965099 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:11.965210 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:11.965217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.966028 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.966031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:11.966675 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:11.966684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.967188 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.967191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:11.967258 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:11.967836 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:11.967849 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:11.970066 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
W20260811 04:42:11.970412 808 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:11.964216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:36539 (local address 127.22.144.190:40435)
0811 04:42:11.964476 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:11.964480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:11.964511 (+ 31us) server_negotiation.cc:407] Connection header received
0811 04:42:11.964668 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:11.964670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:11.964717 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:11.964790 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:11.965335 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.965909 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.966817 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:11.967067 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:11.967164 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:11.970243 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:36539: BlockingRecv error: recv got EOF from 127.22.144.129:36539 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":60,"threads_started":1}
Aug 11 04:42:12 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:12 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:12.181571 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:12.169849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:38791)
0811 04:42:12.170189 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:12.170203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:12.170313 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:12.170571 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:12.170573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:12.170756 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:12.170854 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:12.170859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.171680 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.171682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:12.172241 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:12.172249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.172757 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.172760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:12.172822 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:12.179312 (+ 6490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:12.179326 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:12.181349 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":140,"threads_started":1}
W20260811 04:42:12.181838 808 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:12.169913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:38791 (local address 127.22.144.190:40435)
0811 04:42:12.170080 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:12.170084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:12.170349 (+ 265us) server_negotiation.cc:407] Connection header received
0811 04:42:12.170401 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:12.170403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:12.170452 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:12.170538 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:12.170991 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.171571 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.172382 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.172642 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.172741 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:12.181710 (+ 8969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:38791: BlockingRecv error: recv got EOF from 127.22.144.130:38791 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 04:42:12 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:12 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:12.381294 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:12.374931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:40287)
0811 04:42:12.375241 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:12.375262 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:12.375370 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:12.375693 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:12.375696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:12.375884 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:12.375982 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:12.375987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.376897 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.376899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:12.377468 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:12.377518 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.377998 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.378001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:12.378077 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:12.378618 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:12.378630 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:12.381119 (+ 2489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":150,"threads_started":1}
W20260811 04:42:12.381590 808 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:12.375086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:40287 (local address 127.22.144.190:40435)
0811 04:42:12.375243 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:12.375247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:12.375332 (+ 85us) server_negotiation.cc:407] Connection header received
0811 04:42:12.375452 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:12.375454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:12.375504 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:12.375637 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:12.376127 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.376780 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.377651 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.377886 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.377978 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:12.381308 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:40287: BlockingRecv error: recv got EOF from 127.22.144.131:40287 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 04:42:12 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:12 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:12.978171 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:12.971255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:50947)
0811 04:42:12.971572 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:12.971591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:12.971719 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:12.972019 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:12.972022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:12.972223 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:12.972331 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:12.972337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.973120 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.973124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:12.973892 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:12.973903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.974414 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.974416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:12.974481 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:12.975274 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:12.975290 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:12.977959 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":120,"threads_started":1}
W20260811 04:42:12.978276 813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:12.971375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:50947 (local address 127.22.144.190:40435)
0811 04:42:12.971625 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:12.971629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:12.971673 (+ 44us) server_negotiation.cc:407] Connection header received
0811 04:42:12.971816 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:12.971818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:12.971865 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:12.971952 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:12.972445 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.972992 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.974047 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:12.974298 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:12.974397 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:12.978119 (+ 3722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:50947: BlockingRecv error: recv got EOF from 127.22.144.129:50947 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
Aug 11 04:42:13 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:13 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:13.188584 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:13.182650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:33389)
0811 04:42:13.182954 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:13.182971 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:13.183087 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:13.183478 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:13.183481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:13.183670 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:13.183776 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:13.183782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.184715 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.184718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:13.185285 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:13.185294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.185968 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.185971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:13.186046 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:13.186633 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:13.186644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:13.188459 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":112,"threads_started":1}
W20260811 04:42:13.188917 813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:13.182720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:33389 (local address 127.22.144.190:40435)
0811 04:42:13.182902 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:13.182910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:13.183055 (+ 145us) server_negotiation.cc:407] Connection header received
0811 04:42:13.183225 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:13.183228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:13.183306 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:13.183427 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:13.183915 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.184582 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.185420 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.185841 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.185941 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:13.188637 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:33389: BlockingRecv error: recv got EOF from 127.22.144.130:33389 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 04:42:13 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:13 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:13.388252 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:13.382262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:39701)
0811 04:42:13.382595 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:13.382612 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:13.382730 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:13.382988 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:13.382991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:13.383204 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:13.383307 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:13.383313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.384124 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.384127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:13.384735 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:13.384743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.385193 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.385196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:13.385263 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:13.385853 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:13.385866 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:13.388058 (+ 2192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":124,"threads_started":1}
W20260811 04:42:13.388398 813 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:13.382458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:39701 (local address 127.22.144.190:40435)
0811 04:42:13.382607 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:13.382611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:13.382700 (+ 89us) server_negotiation.cc:407] Connection header received
0811 04:42:13.382816 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:13.382819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:13.382864 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:13.382959 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:13.383430 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.383995 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.384851 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.385086 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.385177 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:13.388255 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:39701: BlockingRecv error: recv got EOF from 127.22.144.131:39701 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 04:42:13 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:13 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:13.985852 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:13.979526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:47519)
0811 04:42:13.979862 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:13.979878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:13.979993 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:13.980363 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:13.980366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:13.980590 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:13.980691 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:13.980720 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.982025 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.982029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:13.982650 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:13.982661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.983161 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.983164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:13.983228 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:13.983836 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:13.983849 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:13.985734 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
W20260811 04:42:13.986096 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:13.979624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:47519 (local address 127.22.144.190:40435)
0811 04:42:13.979891 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:13.979895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:13.979943 (+ 48us) server_negotiation.cc:407] Connection header received
0811 04:42:13.980110 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:13.980114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:13.980162 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:13.980287 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:13.980890 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.981873 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.982811 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:13.983053 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:13.983145 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:13.985928 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:47519: BlockingRecv error: recv got EOF from 127.22.144.129:47519 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":66,"threads_started":1}
Aug 11 04:42:14 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:14 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:14.195052 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:14.189438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:47477)
0811 04:42:14.189858 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:14.189872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:14.189973 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:14.190201 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:14.190203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:14.190388 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:14.190491 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:14.190496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.191298 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.191301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:14.191921 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:14.191930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.192381 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.192383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:14.192449 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:14.193017 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:14.193031 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:14.194922 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":206,"threads_started":1}
W20260811 04:42:14.195245 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:14.189668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:47477 (local address 127.22.144.190:40435)
0811 04:42:14.189821 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:14.189825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:14.189922 (+ 97us) server_negotiation.cc:407] Connection header received
0811 04:42:14.190039 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:14.190043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:14.190088 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:14.190162 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:14.190645 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.191197 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.192048 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.192276 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.192357 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:14.195110 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:47477: BlockingRecv error: recv got EOF from 127.22.144.130:47477 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 04:42:14 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:14 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:14.395180 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:14.389151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:39371)
0811 04:42:14.389542 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:14.389562 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:14.389713 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:14.390001 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:14.390004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:14.390244 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:14.390341 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:14.390346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.391184 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.391189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:14.391766 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:14.391775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.392258 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.392262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:14.392353 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:14.392952 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:14.392964 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:14.395061 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":94,"threads_started":1}
W20260811 04:42:14.395370 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:14.389292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:39371 (local address 127.22.144.190:40435)
0811 04:42:14.389468 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:14.389474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:14.389643 (+ 169us) server_negotiation.cc:407] Connection header received
0811 04:42:14.389818 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:14.389822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:14.389877 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:14.389975 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:14.390467 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.391063 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.391891 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.392125 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.392250 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:14.395208 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:39371: BlockingRecv error: recv got EOF from 127.22.144.131:39371 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 04:42:14 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:14 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:14.992952 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:14.986666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:52665)
0811 04:42:14.987032 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:14.987048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:14.987171 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:14.987445 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:14.987447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:14.987648 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:14.987759 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:14.987773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.988719 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.988723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:14.989360 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:14.989369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.989952 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.989955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:14.990046 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:14.990594 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:14.990608 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:14.992827 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":99,"threads_started":1}
W20260811 04:42:14.993167 822 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:14.986805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:52665 (local address 127.22.144.190:40435)
0811 04:42:14.987057 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:14.987060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:14.987159 (+ 99us) server_negotiation.cc:407] Connection header received
0811 04:42:14.987264 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:14.987266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:14.987311 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:14.987393 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:14.987916 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.988581 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.989566 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:14.989834 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:14.989925 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:14.992994 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:52665: BlockingRecv error: recv got EOF from 127.22.144.129:52665 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":64,"threads_started":1}
Aug 11 04:42:15 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:15 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:15.202461 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:15.196006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:58143)
0811 04:42:15.196329 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:15.196344 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:15.196456 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:15.196688 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:15.196690 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:15.196900 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:15.197006 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:15.197012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.197944 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.197948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:15.198858 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:15.198870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.199497 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.199502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:15.199603 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:15.200246 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:15.200260 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:15.202283 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":116,"threads_started":1}
W20260811 04:42:15.202523 822 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:15.196086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:58143 (local address 127.22.144.190:40435)
0811 04:42:15.196263 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:15.196267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:15.196424 (+ 157us) server_negotiation.cc:407] Connection header received
0811 04:42:15.196538 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:15.196541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:15.196587 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:15.196664 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:15.197146 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.197823 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.199021 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.199369 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.199480 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:15.202413 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:58143: BlockingRecv error: recv got EOF from 127.22.144.130:58143 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 04:42:15 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:15 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:15.402135 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:15.396073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:37725)
0811 04:42:15.396410 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:15.396422 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:15.396519 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:15.396837 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:15.396839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:15.397023 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:15.397123 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:15.397128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.398053 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.398056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:15.398830 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:15.398841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.399325 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.399329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:15.399412 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:15.399944 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:15.399956 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:15.401945 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":84,"threads_started":1}
W20260811 04:42:15.402235 822 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:15.396238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:37725 (local address 127.22.144.190:40435)
0811 04:42:15.396386 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:15.396390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:15.396491 (+ 101us) server_negotiation.cc:407] Connection header received
0811 04:42:15.396666 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:15.396668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:15.396719 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:15.396804 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:15.397300 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.397947 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.398959 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.399206 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.399319 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:15.402078 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:37725: BlockingRecv error: recv got EOF from 127.22.144.131:37725 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 04:42:15 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:15 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:16.000983 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:15.994122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:51695)
0811 04:42:15.994437 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:15.994451 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:15.994547 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:15.995033 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:15.995038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:15.995326 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:15.995489 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:15.995509 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.996436 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.996439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:15.997252 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:15.997268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.997987 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.997993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:15.998083 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:15.998753 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:15.998810 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:16.000854 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":118,"threads_started":1}
W20260811 04:42:16.001139 827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:15.994213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:51695 (local address 127.22.144.190:40435)
0811 04:42:15.994489 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:15.994529 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:15.994544 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:42:15.994742 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:15.994745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:15.994819 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:15.994921 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:15.995669 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.996311 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.997387 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:15.997813 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:15.997973 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:16.000998 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:51695: BlockingRecv error: recv got EOF from 127.22.144.129:51695 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":85,"threads_started":1}
Aug 11 04:42:16 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:16 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:16.209645 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:16.203400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:51559)
0811 04:42:16.203754 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:16.203774 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:16.203895 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:16.204215 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:16.204218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:16.204434 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:16.204535 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:16.204540 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.205348 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.205353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:16.206024 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:16.206034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.206522 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.206526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:16.206619 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:16.207213 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:16.207226 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:16.209443 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":122,"threads_started":1}
W20260811 04:42:16.209795 827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:16.203694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:51559 (local address 127.22.144.190:40435)
0811 04:42:16.203865 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:16.203870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:16.203884 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:42:16.204004 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:16.204007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:16.204059 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:16.204157 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:16.204701 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.205231 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.206164 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.206394 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.206505 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:16.209677 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:51559: BlockingRecv error: recv got EOF from 127.22.144.130:51559 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 04:42:16 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:16 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:16.409296 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:16.403050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:36315)
0811 04:42:16.403442 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:16.403460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:16.403614 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:16.403931 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:16.403934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:16.404191 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:16.404294 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:16.404300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.405104 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.405108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:16.405817 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:16.405832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.406400 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.406408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:16.406527 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:16.407190 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:16.407207 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:16.409178 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":125,"threads_started":1}
W20260811 04:42:16.409454 827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:16.403240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:36315 (local address 127.22.144.190:40435)
0811 04:42:16.403427 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:16.403431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:16.403554 (+ 123us) server_negotiation.cc:407] Connection header received
0811 04:42:16.403741 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:16.403745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:16.403804 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:16.403900 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:16.404429 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.404975 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.405951 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:16.406216 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:16.406333 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:16.409329 (+ 2996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:36315: BlockingRecv error: recv got EOF from 127.22.144.131:36315 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 04:42:17 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:17 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:17.007683 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:17.001908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:53611)
0811 04:42:17.002241 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:17.002259 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:17.002373 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:17.002610 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:17.002613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:17.002806 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:17.002906 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:17.002912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.003677 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.003679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:17.004242 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:17.004250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.004777 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.004779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:17.004846 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:17.005369 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:17.005382 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:17.007507 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":74,"threads_started":1}
W20260811 04:42:17.007851 831 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:17.002026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53611 (local address 127.22.144.190:40435)
0811 04:42:17.002387 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:17.002391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:17.002405 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:42:17.002442 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:17.002444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:17.002485 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:17.002563 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:17.003023 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.003559 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.004377 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.004661 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.004757 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:17.007710 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:53611: BlockingRecv error: recv got EOF from 127.22.144.129:53611 (error 108)
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":97,"threads_started":1}
Aug 11 04:42:17 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:17 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:17.222239 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:17.210750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:34603)
0811 04:42:17.211083 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:17.211097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:17.211246 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:17.211574 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:17.211576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:17.211838 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:17.211938 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:17.211945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.212713 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.212718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:17.213369 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:17.213381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.214036 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.214040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:17.214119 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:17.219544 (+ 5425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:17.219558 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:17.222065 (+ 2507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
W20260811 04:42:17.222502 831 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:17.210833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:34603 (local address 127.22.144.190:40435)
0811 04:42:17.211028 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:17.211033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:17.211185 (+ 152us) server_negotiation.cc:407] Connection header received
0811 04:42:17.211372 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:17.211375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:17.211445 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:17.211538 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:17.212071 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.212599 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.213565 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.213889 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.214001 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:17.222322 (+ 8321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:34603: BlockingRecv error: recv got EOF from 127.22.144.130:34603 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 04:42:17 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:17 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.131@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:17.415884 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:17.410089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.131:41823)
0811 04:42:17.410444 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:17.410457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:17.410660 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:17.410948 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:17.410951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:17.411175 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:17.411273 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:17.411279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.412073 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.412078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:17.412673 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:17.412682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.413158 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.413161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:17.413251 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:17.413790 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:17.413802 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:17.415690 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
W20260811 04:42:17.415932 831 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:17.410218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:41823 (local address 127.22.144.190:40435)
0811 04:42:17.410389 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:17.410393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:17.410537 (+ 144us) server_negotiation.cc:407] Connection header received
0811 04:42:17.410769 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:17.410773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:17.410831 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:17.410919 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:17.411402 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.411950 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.412800 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:17.413038 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:17.413154 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:17.415818 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.131:41823: BlockingRecv error: recv got EOF from 127.22.144.131:41823 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 04:42:18 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:18 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.129@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:18.014698 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:18.008686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.129:51513)
0811 04:42:18.008988 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:18.009003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:18.009113 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:18.009366 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:18.009368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:18.009584 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:18.009731 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:18.009738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.010511 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.010514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:18.011166 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:18.011177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.011672 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.011675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:18.011742 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.012291 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:18.012303 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:18.014572 (+ 2269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":112,"threads_started":1}
W20260811 04:42:18.014925 836 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:18.008734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:51513 (local address 127.22.144.190:40435)
0811 04:42:18.008998 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:18.009001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:18.009099 (+ 98us) server_negotiation.cc:407] Connection header received
0811 04:42:18.009196 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:18.009198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:18.009242 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:18.009325 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:18.009850 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.010399 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.011288 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.011517 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.011613 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.014764 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.129:51513: BlockingRecv error: recv got EOF from 127.22.144.129:51513 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
Aug 11 04:42:18 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 04:42:18 dist-test-slave-ss4n krb5kdc[32103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.144.130@KRBTEST.COM for oryx/127.22.144.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 04:42:18.229420 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:18.223309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:40435 (local address 127.22.144.130:52941)
0811 04:42:18.223673 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:18.223687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:18.223791 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:18.224075 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:18.224077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:18.224262 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:18.224363 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:18.224368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.225428 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.225432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:18.226176 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:18.226187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.226689 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.226692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:18.226776 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.227275 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:18.227286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:18.229236 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.144.190:40435: Server oryx/127.22.144.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":145,"threads_started":1}
W20260811 04:42:18.229512 836 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:18.223414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:52941 (local address 127.22.144.190:40435)
0811 04:42:18.223604 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:18.223610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:18.223766 (+ 156us) server_negotiation.cc:407] Connection header received
0811 04:42:18.223878 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:18.223880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:18.223945 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:18.224025 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:18.224551 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.225313 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.226299 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.226562 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.226682 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.229372 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.144.130:52941: BlockingRecv error: recv got EOF from 127.22.144.130:52941 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 04:42:18.297365 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 302
I20260811 04:42:18.307840 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 471
I20260811 04:42:18.316046 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 621
I20260811 04:42:18.324036 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 32686
2026-08-11T04:42:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18359 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T04:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:42:18Z Disabled control of system clock
I20260811 04:42:18.363122 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:43809
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:43809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 04:42:18.492316 841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:18.492707 841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:18.492753 841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:18.497228 841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:42:18.497349 841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:18.497375 841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:18.497398 841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:42:18.497507 841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:42:18.503639 841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:43809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:43809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:18.504943 841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:18.506569 841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:18.516456 847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:18.516569 841 server_base.cc:1034] running on GCE node
W20260811 04:42:18.516464 850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:18.516435 848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:18.517391 841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:18.518775 841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:18.520081 841 hybrid_clock.cc:648] HybridClock initialized: now 1786423338520061 us; error 40 us; skew 500 ppm
I20260811 04:42:18.522327 841 webserver.cc:460] Webserver started at http://127.22.144.190:43319/ using document root <none> and password file <none>
I20260811 04:42:18.523016 841 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:18.523136 841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:18.523404 841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:18.525800 841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "ec864cdacdcc4281a99b6b9071ab58ed"
format_stamp: "Formatted at 2026-08-11 04:42:18 on dist-test-slave-ss4n"
server_key: "03a1faf7841379725a93bca4d01db464"
server_key_iv: "aeed2cb63925fe566739f5d0e90c254c"
server_key_version: "encryptionkey@0"
I20260811 04:42:18.526427 841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "ec864cdacdcc4281a99b6b9071ab58ed"
format_stamp: "Formatted at 2026-08-11 04:42:18 on dist-test-slave-ss4n"
server_key: "03a1faf7841379725a93bca4d01db464"
server_key_iv: "aeed2cb63925fe566739f5d0e90c254c"
server_key_version: "encryptionkey@0"
I20260811 04:42:18.530633 841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:42:18.533797 856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:18.535328 841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:42:18.535496 841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "ec864cdacdcc4281a99b6b9071ab58ed"
format_stamp: "Formatted at 2026-08-11 04:42:18 on dist-test-slave-ss4n"
server_key: "03a1faf7841379725a93bca4d01db464"
server_key_iv: "aeed2cb63925fe566739f5d0e90c254c"
server_key_version: "encryptionkey@0"
I20260811 04:42:18.535651 841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:18.550419 841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:18.551254 841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:18.551524 841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:18.560316 841 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:43809
I20260811 04:42:18.560323 919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:43809 every 8 connection(s)
I20260811 04:42:18.561447 841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:42:18.564944 920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:18.570963 920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed: Bootstrap starting.
I20260811 04:42:18.571224 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 841
I20260811 04:42:18.571375 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:42:18.571693 23106 external_mini_cluster.cc:1428] Setting key 298bd0ddae39535870b9968efa379e4e
I20260811 04:42:18.573885 920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:18.574677 920 log.cc:824] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:18.576619 920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed: No bootstrap required, opened a new log
I20260811 04:42:18.580154 920 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43809 } }
I20260811 04:42:18.580404 920 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:18.580492 920 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec864cdacdcc4281a99b6b9071ab58ed, State: Initialized, Role: FOLLOWER
I20260811 04:42:18.580978 920 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43809 } }
I20260811 04:42:18.581197 920 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:42:18.581292 920 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:42:18.581399 920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:18.582206 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:18.573203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.22.144.190:43809)
0811 04:42:18.573983 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:18.574026 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:18.574104 (+ 78us) server_negotiation.cc:407] Connection header received
0811 04:42:18.574840 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:18.574864 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:18.575323 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:18.575687 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:18.576158 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.577303 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.578358 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.578897 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.579101 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.580056 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:18.580062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:18.580081 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:18.580556 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:18.580643 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:18.580667 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:18.580785 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:18.580948 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 04:42:18.581275 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":131,"threads_started":1}
I20260811 04:42:18.582475 920 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43809 } }
I20260811 04:42:18.582861 920 leader_election.cc:302] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec864cdacdcc4281a99b6b9071ab58ed; no voters:
I20260811 04:42:18.583194 920 leader_election.cc:288] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:42:18.583536 926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:42:18.583827 926 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [term 1 LEADER]: Becoming Leader. State: Replica: ec864cdacdcc4281a99b6b9071ab58ed, State: Running, Role: LEADER
I20260811 04:42:18.584203 926 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43809 } }
I20260811 04:42:18.584471 920 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:42:18.586055 927 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43809 } } }
I20260811 04:42:18.586195 927 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:18.586489 927 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec864cdacdcc4281a99b6b9071ab58ed. Latest consensus state: current_term: 1 leader_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec864cdacdcc4281a99b6b9071ab58ed" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43809 } } }
I20260811 04:42:18.586570 927 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:18.586642 938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:42:18.589891 938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:42:18.596597 938 catalog_manager.cc:1347] Generated new cluster ID: 5af065cdb8244dd081d5377dbfd417d6
I20260811 04:42:18.596714 938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:42:18.622913 938 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:42:18.623982 938 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:42:18.638230 938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec864cdacdcc4281a99b6b9071ab58ed: Generated new TSK 0
I20260811 04:42:18.639082 938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:42:18.645553 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:43809
--builtin_ntp_servers=127.22.144.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 04:42:18.775493 945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:18.775805 945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:18.775846 945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:18.780146 945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:18.780227 945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:18.780284 945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:42:18.785561 945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:18.786995 945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:18.788400 945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:18.798777 953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:18.798795 950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:18.798826 951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:18.799016 945 server_base.cc:1034] running on GCE node
I20260811 04:42:18.799604 945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:18.800287 945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:18.801548 945 hybrid_clock.cc:648] HybridClock initialized: now 1786423338801522 us; error 33 us; skew 500 ppm
I20260811 04:42:18.803627 945 webserver.cc:460] Webserver started at http://127.22.144.129:33745/ using document root <none> and password file <none>
I20260811 04:42:18.804240 945 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:18.804339 945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:18.804571 945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:18.806782 945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "e09ae3978a654fdf97e06f5c4ea72409"
format_stamp: "Formatted at 2026-08-11 04:42:18 on dist-test-slave-ss4n"
server_key: "df8b4c36fa313dc2180f0d4cfe024949"
server_key_iv: "64d98f338b4957f791085ac2eb2519cc"
server_key_version: "encryptionkey@0"
I20260811 04:42:18.807423 945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "e09ae3978a654fdf97e06f5c4ea72409"
format_stamp: "Formatted at 2026-08-11 04:42:18 on dist-test-slave-ss4n"
server_key: "df8b4c36fa313dc2180f0d4cfe024949"
server_key_iv: "64d98f338b4957f791085ac2eb2519cc"
server_key_version: "encryptionkey@0"
I20260811 04:42:18.811434 945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:42:18.814829 959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:18.816206 945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:42:18.816363 945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "e09ae3978a654fdf97e06f5c4ea72409"
format_stamp: "Formatted at 2026-08-11 04:42:18 on dist-test-slave-ss4n"
server_key: "df8b4c36fa313dc2180f0d4cfe024949"
server_key_iv: "64d98f338b4957f791085ac2eb2519cc"
server_key_version: "encryptionkey@0"
I20260811 04:42:18.816504 945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:18.835614 945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:18.836448 945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:18.836699 945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:18.837387 945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:18.838502 945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:18.838593 945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:18.838697 945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:18.838754 945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:18.849438 1094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:46407 every 8 connection(s)
I20260811 04:42:18.849429 945 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:46407
I20260811 04:42:18.850649 945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:42:18.854826 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 945
I20260811 04:42:18.854974 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:42:18.855311 23106 external_mini_cluster.cc:1428] Setting key f5a1661cd01b17e832252766d4286363
I20260811 04:42:18.857823 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:43809
--builtin_ntp_servers=127.22.144.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 04:42:18.861855 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:18.852738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:36773 (local address 127.22.144.190:43809)
0811 04:42:18.852896 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:18.852900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:18.854185 (+ 1285us) server_negotiation.cc:407] Connection header received
0811 04:42:18.855924 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:18.855931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:18.856000 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:18.856105 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:18.857374 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.858359 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.859596 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.859842 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.859966 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.861180 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:18.861187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:18.861193 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:18.861211 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:18.861237 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:18.861250 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:18.861325 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:18.861563 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 04:42:18.861641 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:42:18.862751 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:18.853241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:43809 (local address 127.22.144.129:36773)
0811 04:42:18.854042 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:18.854087 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:18.855614 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:18.856323 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:18.856332 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:18.856443 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:18.857206 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:18.857224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.858528 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.858531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:18.859455 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:18.859466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:18.860012 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:18.860016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:18.860189 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:18.860900 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:42:18.860901 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:18.860943 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:42:18.861008 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:18.861358 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:18.861368 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:18.861374 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:42:18.861551 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 04:42:18.861801 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":125,"threads_started":1}
I20260811 04:42:18.864178 1095 heartbeater.cc:347] Connected to a master server at 127.22.144.190:43809
I20260811 04:42:18.864450 1095 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:18.865037 1095 heartbeater.cc:511] Master 127.22.144.190:43809 requested a full tablet report, sending...
I20260811 04:42:18.866976 876 ts_manager.cc:194] Registered new tserver with Master: e09ae3978a654fdf97e06f5c4ea72409 (127.22.144.129:46407)
I20260811 04:42:18.868822 876 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.129:36773
W20260811 04:42:18.975734 1098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:18.976012 1098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:18.976058 1098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:18.980041 1098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:18.980113 1098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:18.980167 1098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:42:18.985124 1098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:18.986377 1098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:18.987840 1098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:18.998034 1105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:18.998068 1104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:18.998333 1098 server_base.cc:1034] running on GCE node
W20260811 04:42:18.998059 1107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:18.998924 1098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:18.999614 1098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:19.000821 1098 hybrid_clock.cc:648] HybridClock initialized: now 1786423339000806 us; error 27 us; skew 500 ppm
I20260811 04:42:19.002825 1098 webserver.cc:460] Webserver started at http://127.22.144.130:45401/ using document root <none> and password file <none>
I20260811 04:42:19.003443 1098 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:19.003541 1098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:19.003783 1098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:19.005996 1098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "c3e3457e1b12472b8ecb79d9b5e4f54f"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "b21e03290b8a2fd083d1502366c4f386"
server_key_iv: "52b2599dc62c35d269323a3b7c1e90d5"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.006598 1098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "c3e3457e1b12472b8ecb79d9b5e4f54f"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "b21e03290b8a2fd083d1502366c4f386"
server_key_iv: "52b2599dc62c35d269323a3b7c1e90d5"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.010514 1098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:42:19.013247 1113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.014720 1098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:42:19.014866 1098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "c3e3457e1b12472b8ecb79d9b5e4f54f"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "b21e03290b8a2fd083d1502366c4f386"
server_key_iv: "52b2599dc62c35d269323a3b7c1e90d5"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.015002 1098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:19.043792 1098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:19.044598 1098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:19.044829 1098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:19.045465 1098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:19.046598 1098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:19.046669 1098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.046744 1098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:19.046797 1098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.056232 1098 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:43283
I20260811 04:42:19.056258 1241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:43283 every 8 connection(s)
I20260811 04:42:19.057302 1098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:42:19.066762 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1098
I20260811 04:42:19.066902 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:42:19.067243 23106 external_mini_cluster.cc:1428] Setting key 9834290321a005faa9fb7a094ceed9ac
I20260811 04:42:19.067317 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.059369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:39861 (local address 127.22.144.190:43809)
0811 04:42:19.059528 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:19.059532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:19.060691 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 04:42:19.062233 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:19.062237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:19.062300 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:19.062384 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:19.063315 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.063868 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.065052 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.065289 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.065409 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:19.066735 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:19.066739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:19.066741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:19.066760 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:19.066781 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:19.066790 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:19.066917 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:19.067114 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 04:42:19.067175 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 04:42:19.068392 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.059792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:43809 (local address 127.22.144.130:39861)
0811 04:42:19.060527 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:19.060570 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:19.061986 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:19.062563 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:19.062570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:19.062658 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:19.063160 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:19.063173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.064016 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.064020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:19.064915 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:19.064926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.065457 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.065462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:19.065666 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:19.066368 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:42:19.066369 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:19.066435 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:42:19.066533 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:19.066934 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:19.066947 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:19.066956 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:42:19.067113 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 04:42:19.067432 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":141,"threads_started":1}
I20260811 04:42:19.069722 1242 heartbeater.cc:347] Connected to a master server at 127.22.144.190:43809
I20260811 04:42:19.069962 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:43809
--builtin_ntp_servers=127.22.144.148:35421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 04:42:19.069999 1242 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:19.070564 1242 heartbeater.cc:511] Master 127.22.144.190:43809 requested a full tablet report, sending...
I20260811 04:42:19.071767 876 ts_manager.cc:194] Registered new tserver with Master: c3e3457e1b12472b8ecb79d9b5e4f54f (127.22.144.130:43283)
I20260811 04:42:19.072391 876 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.130:39861
W20260811 04:42:19.191453 1245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:19.191758 1245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:19.191800 1245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:19.196086 1245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:19.196156 1245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:19.196311 1245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:42:19.202111 1245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:19.203363 1245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:19.204979 1245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:19.215387 1251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.215426 1250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.215426 1253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:19.215806 1245 server_base.cc:1034] running on GCE node
I20260811 04:42:19.216336 1245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:19.217115 1245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:19.218326 1245 hybrid_clock.cc:648] HybridClock initialized: now 1786423339218306 us; error 48 us; skew 500 ppm
I20260811 04:42:19.220541 1245 webserver.cc:460] Webserver started at http://127.22.144.131:43133/ using document root <none> and password file <none>
I20260811 04:42:19.221172 1245 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:19.221239 1245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:19.221470 1245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:19.223968 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "7cfe8c5d710740948e4d30e8995faef4"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "3398c29e1deae3d3fd26f0fea6e9a05e"
server_key_iv: "30592e7007192479aebd843f01a057b9"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.224630 1245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "7cfe8c5d710740948e4d30e8995faef4"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "3398c29e1deae3d3fd26f0fea6e9a05e"
server_key_iv: "30592e7007192479aebd843f01a057b9"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.228855 1245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 04:42:19.231732 1259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.233059 1245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:42:19.233191 1245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "7cfe8c5d710740948e4d30e8995faef4"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "3398c29e1deae3d3fd26f0fea6e9a05e"
server_key_iv: "30592e7007192479aebd843f01a057b9"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.233314 1245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:19.262295 1245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:19.263149 1245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:19.263386 1245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:19.264091 1245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:19.265162 1245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:19.265215 1245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.265260 1245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:19.265319 1245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.275322 1245 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:35283
I20260811 04:42:19.275352 1388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:35283 every 8 connection(s)
I20260811 04:42:19.276394 1245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:42:19.278851 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1245
I20260811 04:42:19.278968 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:42:19.279318 23106 external_mini_cluster.cc:1428] Setting key 19b2e8b437c0c9f9d70cdad48cc38a74
I20260811 04:42:19.288305 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.278408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:34015 (local address 127.22.144.190:43809)
0811 04:42:19.278690 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:19.278694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:19.280031 (+ 1337us) server_negotiation.cc:407] Connection header received
0811 04:42:19.281901 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:19.281904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:19.281964 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:19.282049 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:19.283039 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.283652 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.285761 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.286230 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.286372 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:19.287724 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:19.287728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:19.287731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:19.287746 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:19.287775 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:19.287785 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:19.287872 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:19.288076 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 04:42:19.288146 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260811 04:42:19.289299 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.278957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:43809 (local address 127.22.144.131:34015)
0811 04:42:19.279851 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:19.279908 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:19.281683 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:19.282257 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:19.282265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:19.282358 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:19.282878 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:19.282891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.283822 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.283825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:19.285572 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:19.285587 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:19.286452 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:19.286460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:19.286736 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:19.287437 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:42:19.287439 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:19.287484 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:42:19.287556 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:19.287875 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:19.287888 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:19.287896 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 04:42:19.288063 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 04:42:19.288411 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":147,"threads_started":1}
I20260811 04:42:19.290549 1389 heartbeater.cc:347] Connected to a master server at 127.22.144.190:43809
I20260811 04:42:19.290817 1389 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:19.291391 1389 heartbeater.cc:511] Master 127.22.144.190:43809 requested a full tablet report, sending...
I20260811 04:42:19.292666 875 ts_manager.cc:194] Registered new tserver with Master: 7cfe8c5d710740948e4d30e8995faef4 (127.22.144.131:35283)
I20260811 04:42:19.293412 875 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.131:34015
I20260811 04:42:19.293694 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 04:42:19.296885 923 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:19.296044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.22.144.190:43809)
0811 04:42:19.296166 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:19.296170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:19.296360 (+ 190us) server_negotiation.cc:407] Connection header received
0811 04:42:19.296445 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:19.296448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:19.296487 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:19.296560 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:19.296784 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42444: BlockingRecv error: recv got EOF from 127.0.0.1:42444 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 04:42:19.297300 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 945
I20260811 04:42:19.311676 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1098
I20260811 04:42:19.321435 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1245
I20260811 04:42:19.330766 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 841
2026-08-11T04:42:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1004 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T04:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:42:19Z Disabled control of system clock
I20260811 04:42:19.365864 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:34027
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:34027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 04:42:19.486572 1400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:19.486857 1400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:19.486896 1400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:19.491016 1400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:42:19.491089 1400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:19.491106 1400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:19.491123 1400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:42:19.491139 1400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:42:19.496011 1400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:34027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:34027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:19.497109 1400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:19.498603 1400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:19.507414 1407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.507440 1409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.507488 1406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:19.507718 1400 server_base.cc:1034] running on GCE node
I20260811 04:42:19.508529 1400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:19.509809 1400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:19.511023 1400 hybrid_clock.cc:648] HybridClock initialized: now 1786423339510994 us; error 54 us; skew 500 ppm
I20260811 04:42:19.513222 1400 webserver.cc:460] Webserver started at http://127.22.144.190:32889/ using document root <none> and password file <none>
I20260811 04:42:19.513923 1400 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:19.514029 1400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:19.514274 1400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:19.516785 1400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "5dab221419874849a624cbd70f4c7ea3"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "cdcfe8bf66f7b29dc53526b893dc35da"
server_key_iv: "01fd3803990aec4f6e671669211fd6c7"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.517552 1400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "5dab221419874849a624cbd70f4c7ea3"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "cdcfe8bf66f7b29dc53526b893dc35da"
server_key_iv: "01fd3803990aec4f6e671669211fd6c7"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.521968 1400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:42:19.524884 1415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.526458 1400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:19.526635 1400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "5dab221419874849a624cbd70f4c7ea3"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "cdcfe8bf66f7b29dc53526b893dc35da"
server_key_iv: "01fd3803990aec4f6e671669211fd6c7"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.526769 1400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:19.531733 1400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:19.532378 1400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:19.532572 1400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:19.539976 1400 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:34027
I20260811 04:42:19.540001 1471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:34027 every 8 connection(s)
I20260811 04:42:19.540951 1400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:42:19.542855 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1400
I20260811 04:42:19.543009 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:42:19.543373 23106 external_mini_cluster.cc:1428] Setting key e7e5c2954cdd98b7ef1f0c92b9f61ff0
I20260811 04:42:19.544993 1472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:19.550110 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.545103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.22.144.190:34027)
0811 04:42:19.546018 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:19.546035 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:19.546076 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:42:19.546830 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:19.546848 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:19.547249 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:19.547646 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:19.548613 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:19.548618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:19.548632 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:19.548644 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:19.548706 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:19.548723 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:19.548900 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:19.548998 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 04:42:19.549264 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":159,"threads_started":1}
I20260811 04:42:19.551740 1472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3: Bootstrap starting.
I20260811 04:42:19.554440 1472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:19.555465 1472 log.cc:824] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:19.557978 1472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3: No bootstrap required, opened a new log
I20260811 04:42:19.562134 1472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dab221419874849a624cbd70f4c7ea3" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34027 } }
I20260811 04:42:19.562510 1472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:19.562604 1472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dab221419874849a624cbd70f4c7ea3, State: Initialized, Role: FOLLOWER
I20260811 04:42:19.563038 1472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dab221419874849a624cbd70f4c7ea3" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34027 } }
I20260811 04:42:19.563166 1472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:42:19.563220 1472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:42:19.563301 1472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:19.564554 1472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dab221419874849a624cbd70f4c7ea3" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34027 } }
I20260811 04:42:19.564895 1472 leader_election.cc:302] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dab221419874849a624cbd70f4c7ea3; no voters:
I20260811 04:42:19.565217 1472 leader_election.cc:288] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:42:19.565414 1478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:42:19.565881 1478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 5dab221419874849a624cbd70f4c7ea3, State: Running, Role: LEADER
I20260811 04:42:19.566257 1478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dab221419874849a624cbd70f4c7ea3" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34027 } }
I20260811 04:42:19.566763 1472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:42:19.568127 1479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dab221419874849a624cbd70f4c7ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dab221419874849a624cbd70f4c7ea3" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34027 } } }
I20260811 04:42:19.568296 1479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:19.568712 1480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dab221419874849a624cbd70f4c7ea3. Latest consensus state: current_term: 1 leader_uuid: "5dab221419874849a624cbd70f4c7ea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dab221419874849a624cbd70f4c7ea3" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 34027 } } }
I20260811 04:42:19.568825 1480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3 [sys.catalog]: This master's current role is: LEADER
W20260811 04:42:19.572009 1494 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:42:19.572084 1494 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:42:19.572176 1489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:42:19.572875 1489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:42:19.578797 1489 catalog_manager.cc:1347] Generated new cluster ID: 011948f1455c4aff830e56fdf7c17343
I20260811 04:42:19.578874 1489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:42:19.587357 1489 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:42:19.587668 1489 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:42:19.596261 1489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dab221419874849a624cbd70f4c7ea3: Generated new TSK 0
I20260811 04:42:19.597071 1489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:42:19.605672 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34027
--builtin_ntp_servers=127.22.144.148:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 04:42:19.746783 1498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:19.747076 1498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:19.747124 1498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:19.751348 1498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:19.751415 1498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:19.751474 1498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:42:19.756559 1498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:19.757699 1498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:19.759105 1498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:19.770169 1505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.770182 1506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.770155 1508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:19.770696 1498 server_base.cc:1034] running on GCE node
I20260811 04:42:19.771220 1498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:19.771981 1498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:19.773193 1498 hybrid_clock.cc:648] HybridClock initialized: now 1786423339773180 us; error 28 us; skew 500 ppm
I20260811 04:42:19.775478 1498 webserver.cc:460] Webserver started at http://127.22.144.129:39211/ using document root <none> and password file <none>
I20260811 04:42:19.776136 1498 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:19.776240 1498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:19.776499 1498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:19.778841 1498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "6167bd889fcc4f408b0d81a07820797c"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "24045b994a2090ef89dec73afc679c50"
server_key_iv: "f54e034d9e5dcefc3139274a28181ebe"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.779464 1498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "6167bd889fcc4f408b0d81a07820797c"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "24045b994a2090ef89dec73afc679c50"
server_key_iv: "f54e034d9e5dcefc3139274a28181ebe"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.783596 1498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:42:19.786612 1514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.787981 1498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:19.788136 1498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "6167bd889fcc4f408b0d81a07820797c"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "24045b994a2090ef89dec73afc679c50"
server_key_iv: "f54e034d9e5dcefc3139274a28181ebe"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.788288 1498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:19.793823 1498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:19.794559 1498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:19.794770 1498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:19.795367 1498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:19.796361 1498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:19.796434 1498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.796502 1498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:19.796550 1498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.806417 1498 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:43367
I20260811 04:42:19.806438 1642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:43367 every 8 connection(s)
I20260811 04:42:19.807405 1498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:42:19.813897 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.809379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:60761 (local address 127.22.144.190:34027)
0811 04:42:19.809580 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:19.809585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:19.810754 (+ 1169us) server_negotiation.cc:407] Connection header received
0811 04:42:19.812087 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:19.812090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:19.812132 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:19.812225 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:19.813426 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:19.813430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:19.813432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:19.813456 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:19.813474 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:19.813482 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:19.813622 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:19.813745 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 04:42:19.813784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:42:19.814834 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:19.809867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:34027 (local address 127.22.144.129:60761)
0811 04:42:19.810582 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:19.810633 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:19.811879 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:19.812390 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:19.812397 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:19.812488 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:19.813162 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:42:19.813163 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:19.813205 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:42:19.813276 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:19.813610 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:19.813618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:19.813623 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 04:42:19.813764 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 04:42:19.813996 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
I20260811 04:42:19.815059 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1498
I20260811 04:42:19.815177 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:42:19.815529 23106 external_mini_cluster.cc:1428] Setting key 0e2e71b3600abac5a3f4ed10d64db67a
I20260811 04:42:19.816319 1644 heartbeater.cc:347] Connected to a master server at 127.22.144.190:34027
I20260811 04:42:19.816573 1644 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:19.817144 1644 heartbeater.cc:511] Master 127.22.144.190:34027 requested a full tablet report, sending...
I20260811 04:42:19.817830 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34027
--builtin_ntp_servers=127.22.144.148:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 04:42:19.818894 1432 ts_manager.cc:194] Registered new tserver with Master: 6167bd889fcc4f408b0d81a07820797c (127.22.144.129:43367)
W20260811 04:42:19.940451 1647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:19.940797 1647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:19.940888 1647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:19.945226 1647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:19.945319 1647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:19.945451 1647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:42:19.950714 1647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:19.951849 1647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:19.953239 1647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:19.963471 1654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.963388 1656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:19.963534 1653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:19.963820 1647 server_base.cc:1034] running on GCE node
I20260811 04:42:19.964241 1647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:19.964876 1647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:19.966073 1647 hybrid_clock.cc:648] HybridClock initialized: now 1786423339966030 us; error 50 us; skew 500 ppm
I20260811 04:42:19.968397 1647 webserver.cc:460] Webserver started at http://127.22.144.130:39393/ using document root <none> and password file <none>
I20260811 04:42:19.969043 1647 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:19.969110 1647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:19.969349 1647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:19.971550 1647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "e57aad5bf2264b5f8ca5b26371262136"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "88cf4bab2d8791a554cf5fc6851240d9"
server_key_iv: "8419d5997aa6be543886a64ff18e9125"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.972154 1647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "e57aad5bf2264b5f8ca5b26371262136"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "88cf4bab2d8791a554cf5fc6851240d9"
server_key_iv: "8419d5997aa6be543886a64ff18e9125"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.976255 1647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:42:19.979152 1662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.980518 1647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:42:19.980669 1647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "e57aad5bf2264b5f8ca5b26371262136"
format_stamp: "Formatted at 2026-08-11 04:42:19 on dist-test-slave-ss4n"
server_key: "88cf4bab2d8791a554cf5fc6851240d9"
server_key_iv: "8419d5997aa6be543886a64ff18e9125"
server_key_version: "encryptionkey@0"
I20260811 04:42:19.980798 1647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:19.985935 1647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:19.986707 1647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:19.986913 1647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:19.987534 1647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:19.988533 1647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:19.988606 1647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.988672 1647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:19.988722 1647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:19.998255 1647 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:37969
I20260811 04:42:19.998277 1786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:37969 every 8 connection(s)
I20260811 04:42:19.999562 1647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:42:20.006557 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:20.001817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:43829 (local address 127.22.144.190:34027)
0811 04:42:20.001965 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:20.001987 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:20.003245 (+ 1258us) server_negotiation.cc:407] Connection header received
0811 04:42:20.004662 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:20.004665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:20.004707 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:20.004790 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:20.005996 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:20.006000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:20.006002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:20.006015 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:20.006037 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:20.006045 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:20.006136 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:20.006319 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 04:42:20.006374 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:42:20.006955 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1647
I20260811 04:42:20.007098 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:42:20.007452 23106 external_mini_cluster.cc:1428] Setting key a2e5618107adbb8f7ee575ecaf386af3
I20260811 04:42:20.007777 1789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:20.002228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:34027 (local address 127.22.144.130:43829)
0811 04:42:20.003057 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:20.003101 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:20.004372 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:20.004925 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:20.004933 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:20.005031 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:20.005688 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:42:20.005690 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:20.005733 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:42:20.005805 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:20.006166 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:20.006175 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:20.006181 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:42:20.006343 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 04:42:20.006591 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":174,"threads_started":1}
I20260811 04:42:20.009280 1787 heartbeater.cc:347] Connected to a master server at 127.22.144.190:34027
I20260811 04:42:20.009598 1787 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:20.010007 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34027
--builtin_ntp_servers=127.22.144.148:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 04:42:20.010159 1787 heartbeater.cc:511] Master 127.22.144.190:34027 requested a full tablet report, sending...
I20260811 04:42:20.011493 1432 ts_manager.cc:194] Registered new tserver with Master: e57aad5bf2264b5f8ca5b26371262136 (127.22.144.130:37969)
W20260811 04:42:20.130800 1790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:20.131179 1790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:20.131282 1790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:20.135623 1790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:20.135720 1790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:20.135823 1790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:42:20.140899 1790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:34027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:20.142205 1790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:20.143677 1790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:20.153888 1797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:20.153954 1790 server_base.cc:1034] running on GCE node
W20260811 04:42:20.153889 1801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:20.153900 1796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:20.154639 1790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:20.155296 1790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:20.156481 1790 hybrid_clock.cc:648] HybridClock initialized: now 1786423340156487 us; error 74 us; skew 500 ppm
I20260811 04:42:20.158668 1790 webserver.cc:460] Webserver started at http://127.22.144.131:40907/ using document root <none> and password file <none>
I20260811 04:42:20.159325 1790 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:20.159397 1790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:20.159664 1790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:20.162111 1790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "797d146219e24ed3a611411d867f7cc4"
format_stamp: "Formatted at 2026-08-11 04:42:20 on dist-test-slave-ss4n"
server_key: "1abf29d95819e04f1d5c107fc6d3f4c4"
server_key_iv: "58d6505957a0236c4ef70ad4f7c841f8"
server_key_version: "encryptionkey@0"
I20260811 04:42:20.162811 1790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "797d146219e24ed3a611411d867f7cc4"
format_stamp: "Formatted at 2026-08-11 04:42:20 on dist-test-slave-ss4n"
server_key: "1abf29d95819e04f1d5c107fc6d3f4c4"
server_key_iv: "58d6505957a0236c4ef70ad4f7c841f8"
server_key_version: "encryptionkey@0"
I20260811 04:42:20.167152 1790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:42:20.170328 1808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:20.171957 1790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:42:20.172113 1790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "797d146219e24ed3a611411d867f7cc4"
format_stamp: "Formatted at 2026-08-11 04:42:20 on dist-test-slave-ss4n"
server_key: "1abf29d95819e04f1d5c107fc6d3f4c4"
server_key_iv: "58d6505957a0236c4ef70ad4f7c841f8"
server_key_version: "encryptionkey@0"
I20260811 04:42:20.172254 1790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:20.177656 1790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:20.178411 1790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:20.178627 1790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:20.179255 1790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:20.180285 1790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:20.180336 1790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:20.180413 1790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:20.180457 1790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:20.190539 1790 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:33689
I20260811 04:42:20.190562 1936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:33689 every 8 connection(s)
I20260811 04:42:20.191589 1790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:42:20.198287 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:20.193613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:59875 (local address 127.22.144.190:34027)
0811 04:42:20.193821 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:20.193826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:20.195138 (+ 1312us) server_negotiation.cc:407] Connection header received
0811 04:42:20.196456 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:20.196459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:20.196503 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:20.196597 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:20.197850 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:20.197853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:20.197855 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:42:20.197869 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:20.197890 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:42:20.197899 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:20.197989 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:20.198123 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 04:42:20.198168 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 04:42:20.198746 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1790
I20260811 04:42:20.198879 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:42:20.199204 23106 external_mini_cluster.cc:1428] Setting key 309503f37233ca6537763a55ecf9deee
I20260811 04:42:20.199362 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:20.194254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:34027 (local address 127.22.144.131:59875)
0811 04:42:20.194964 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:20.195005 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:20.196231 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:20.196768 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:20.196775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:20.196886 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:20.197576 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:42:20.197578 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:20.197620 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:42:20.197694 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:20.197984 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:20.197990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:20.197995 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 04:42:20.198134 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 04:42:20.198383 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":139,"threads_started":1}
I20260811 04:42:20.200878 1939 heartbeater.cc:347] Connected to a master server at 127.22.144.190:34027
I20260811 04:42:20.201139 1939 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:20.201663 1939 heartbeater.cc:511] Master 127.22.144.190:34027 requested a full tablet report, sending...
I20260811 04:42:20.202824 1431 ts_manager.cc:194] Registered new tserver with Master: 797d146219e24ed3a611411d867f7cc4 (127.22.144.131:33689)
I20260811 04:42:20.213954 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 04:42:20.217247 1476 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:42:20.216150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.22.144.190:34027)
0811 04:42:20.216328 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:20.216333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:20.216484 (+ 151us) server_negotiation.cc:407] Connection header received
0811 04:42:20.216662 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:20.216665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:20.216706 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:20.216811 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:20.217084 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53218: BlockingRecv error: recv got EOF from 127.0.0.1:53218 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 04:42:20.217374 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1498
I20260811 04:42:20.227907 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1647
I20260811 04:42:20.237247 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1790
I20260811 04:42:20.245172 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1400
2026-08-11T04:42:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (915 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:42:20 dist-test-slave-ss4n krb5kdc[1950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:42:20 dist-test-slave-ss4n krb5kdc[1950](info): set up 2 sockets
Aug 11 04:42:20 dist-test-slave-ss4n krb5kdc[1950](info): commencing operation
krb5kdc: starting...
W20260811 04:42:21.430683 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:42:21 dist-test-slave-ss4n krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:42:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:42:21.578940 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:36375
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:21.697892 1966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:21.698261 1966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:21.698347 1966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:21.702564 1966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:42:21.702679 1966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:21.702720 1966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:21.702755 1966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:42:21.702788 1966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:42:21.708310 1966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:36375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:21.709547 1966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:21.710636 1966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:21.719730 1971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:21.719707 1974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:21.719707 1972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:21.719760 1966 server_base.cc:1034] running on GCE node
I20260811 04:42:21.720503 1966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:21.721772 1966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:21.722944 1966 hybrid_clock.cc:648] HybridClock initialized: now 1786423341722928 us; error 29 us; skew 500 ppm
Aug 11 04:42:21 dist-test-slave-ss4n krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423341, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:21.725896 1966 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:42:21.726833 1966 webserver.cc:460] Webserver started at http://127.22.144.190:41803/ using document root <none> and password file <none>
I20260811 04:42:21.727367 1966 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:21.727427 1966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:21.727595 1966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:21.729718 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "1f8cf5cf8bd846729599ae66bdd22314"
format_stamp: "Formatted at 2026-08-11 04:42:21 on dist-test-slave-ss4n"
server_key: "195aa8345c52a645453f0f1edba6c80a"
server_key_iv: "c739ed2f2441fd9921f779cd8a3d270b"
server_key_version: "encryptionkey@0"
I20260811 04:42:21.730275 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "1f8cf5cf8bd846729599ae66bdd22314"
format_stamp: "Formatted at 2026-08-11 04:42:21 on dist-test-slave-ss4n"
server_key: "195aa8345c52a645453f0f1edba6c80a"
server_key_iv: "c739ed2f2441fd9921f779cd8a3d270b"
server_key_version: "encryptionkey@0"
I20260811 04:42:21.734151 1966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:42:21.736725 1982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:21.738315 1966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:21.738482 1966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "1f8cf5cf8bd846729599ae66bdd22314"
format_stamp: "Formatted at 2026-08-11 04:42:21 on dist-test-slave-ss4n"
server_key: "195aa8345c52a645453f0f1edba6c80a"
server_key_iv: "c739ed2f2441fd9921f779cd8a3d270b"
server_key_version: "encryptionkey@0"
I20260811 04:42:21.738613 1966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:21.751412 1966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:21.755091 1966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:21.755318 1966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:21.763702 1966 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:36375
I20260811 04:42:21.763780 2041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:36375 every 8 connection(s)
I20260811 04:42:21.764813 1966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:42:21.768005 2042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:21.768733 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 1966
I20260811 04:42:21.768869 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:42:21.769204 23106 external_mini_cluster.cc:1428] Setting key 3370821e76788c6f6f152534f18ce220
I20260811 04:42:21.775180 2042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314: Bootstrap starting.
I20260811 04:42:21.778201 2042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:21.779269 2042 log.cc:824] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:42:21 dist-test-slave-ss4n krb5kdc[1950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:42:21.781594 2042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314: No bootstrap required, opened a new log
I20260811 04:42:21.786310 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:21.770897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.22.144.190:36375)
0811 04:42:21.771382 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:21.771400 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:21.771440 (+ 40us) server_negotiation.cc:407] Connection header received
0811 04:42:21.772203 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:21.772223 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:21.772614 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:21.772921 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:21.773920 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:21.775147 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:21.776321 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:21.776912 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:21.777083 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:21.780525 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:21.780531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:21.780547 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:21.780576 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:21.782808 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:21.783282 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:21.783289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:21.783295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:21.783444 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:21.783786 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:21.783791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:21.783795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:21.784426 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:21.784635 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:21.784894 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 04:42:21.785228 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":163,"threads_started":1}
I20260811 04:42:21.786512 2042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8cf5cf8bd846729599ae66bdd22314" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36375 } }
I20260811 04:42:21.786808 2042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:21.786867 2042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f8cf5cf8bd846729599ae66bdd22314, State: Initialized, Role: FOLLOWER
I20260811 04:42:21.787348 2042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8cf5cf8bd846729599ae66bdd22314" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36375 } }
I20260811 04:42:21.787526 2042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:42:21.787572 2042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:42:21.787690 2042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:21.788741 2042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8cf5cf8bd846729599ae66bdd22314" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36375 } }
I20260811 04:42:21.789115 2042 leader_election.cc:302] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f8cf5cf8bd846729599ae66bdd22314; no voters:
I20260811 04:42:21.789453 2042 leader_election.cc:288] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:42:21.789773 2047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:42:21.790004 2047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [term 1 LEADER]: Becoming Leader. State: Replica: 1f8cf5cf8bd846729599ae66bdd22314, State: Running, Role: LEADER
I20260811 04:42:21.790328 2047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8cf5cf8bd846729599ae66bdd22314" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36375 } }
I20260811 04:42:21.790828 2042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:42:21.792114 2049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f8cf5cf8bd846729599ae66bdd22314. Latest consensus state: current_term: 1 leader_uuid: "1f8cf5cf8bd846729599ae66bdd22314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8cf5cf8bd846729599ae66bdd22314" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36375 } } }
I20260811 04:42:21.792259 2049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:21.792311 2048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f8cf5cf8bd846729599ae66bdd22314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8cf5cf8bd846729599ae66bdd22314" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36375 } } }
I20260811 04:42:21.792421 2048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314 [sys.catalog]: This master's current role is: LEADER
I20260811 04:42:21.792750 2058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:42:21.795509 2058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:42:21.801235 2058 catalog_manager.cc:1347] Generated new cluster ID: 441c5015a28f48898cfb43804516fc9f
I20260811 04:42:21.801328 2058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:42:21.809218 2058 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:42:21.810151 2058 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:42:21.817566 2058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314: Generated new TSK 0
I20260811 04:42:21.818326 2058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:42:21.861675 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36375
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:21.987128 2070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:21.987399 2070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:21.987442 2070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:21.991735 2070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:21.991798 2070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:21.991904 2070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:42:21.997392 2070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:21.998575 2070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:21.999754 2070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:22.010684 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:22.010684 2081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:22.010699 2076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:22.011118 2070 server_base.cc:1034] running on GCE node
I20260811 04:42:22.011627 2070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:22.012367 2070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:22.013643 2070 hybrid_clock.cc:648] HybridClock initialized: now 1786423342013615 us; error 33 us; skew 500 ppm
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:22.016888 2070 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:42:22.018018 2070 webserver.cc:460] Webserver started at http://127.22.144.129:44709/ using document root <none> and password file <none>
I20260811 04:42:22.018612 2070 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:22.018709 2070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:22.018965 2070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:22.021117 2070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "588d418310e74fd9a9dd124bc9c26f1a"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "054157040a820dd263a5428c26eb309f"
server_key_iv: "5725628d3e8ee6534f51ecc52d45b30a"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.021765 2070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "588d418310e74fd9a9dd124bc9c26f1a"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "054157040a820dd263a5428c26eb309f"
server_key_iv: "5725628d3e8ee6534f51ecc52d45b30a"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.025787 2070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:42:22.028386 2089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.029846 2070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:22.029999 2070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "588d418310e74fd9a9dd124bc9c26f1a"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "054157040a820dd263a5428c26eb309f"
server_key_iv: "5725628d3e8ee6534f51ecc52d45b30a"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.030122 2070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:22.041797 2070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:22.045805 2070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:22.046062 2070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:22.046736 2070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:22.047623 2070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:22.047694 2070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.047771 2070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:22.047814 2070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.057253 2070 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:43859
I20260811 04:42:22.057273 2219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:43859 every 8 connection(s)
I20260811 04:42:22.058214 2070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:42:22.062031 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 2070
I20260811 04:42:22.062203 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:42:22.062500 23106 external_mini_cluster.cc:1428] Setting key 2f6b7d2e20a827f8498f68a60cc11ab5
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[1950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:42:22.075661 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.060136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:33901 (local address 127.22.144.190:36375)
0811 04:42:22.060322 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.060327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.061217 (+ 890us) server_negotiation.cc:407] Connection header received
0811 04:42:22.062835 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.062838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.062903 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.063000 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:22.064583 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.065450 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.067476 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.067757 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.067847 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.071280 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:22.071284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:22.071309 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:22.071346 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:22.073127 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.073800 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.073804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.073806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.073897 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.074322 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.074325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.074327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.074596 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:22.074713 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.075271 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.075426 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 04:42:22.076508 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.060562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36375 (local address 127.22.144.129:33901)
0811 04:42:22.061051 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.061096 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.062566 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.063240 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.063247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.063774 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:22.064421 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.064436 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.065679 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.065683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.067314 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.067333 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.067945 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.067953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.068229 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.068938 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:22.068967 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:22.071082 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:22.073273 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:22.073280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:22.073297 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:22.073688 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:22.074047 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:22.074051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:22.074053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:22.074211 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:22.074713 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:22.074722 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:22.075012 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.075214 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.075604 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:42:22.078320 2220 heartbeater.cc:347] Connected to a master server at 127.22.144.190:36375
I20260811 04:42:22.078621 2220 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:22.079347 2220 heartbeater.cc:511] Master 127.22.144.190:36375 requested a full tablet report, sending...
I20260811 04:42:22.081363 2000 ts_manager.cc:194] Registered new tserver with Master: 588d418310e74fd9a9dd124bc9c26f1a (127.22.144.129:43859)
I20260811 04:42:22.083212 2000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:33901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:42:22.097975 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36375
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:22.222249 2225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:22.222558 2225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:22.222613 2225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:22.226807 2225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:22.226871 2225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:22.226958 2225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:42:22.232996 2225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:22.234266 2225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:22.235402 2225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:22.245661 2232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:22.245719 2234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:22.245930 2231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:22.246049 2225 server_base.cc:1034] running on GCE node
I20260811 04:42:22.246482 2225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:22.247164 2225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:22.248387 2225 hybrid_clock.cc:648] HybridClock initialized: now 1786423342248346 us; error 49 us; skew 500 ppm
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:22.256072 2225 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:42:22.257081 2225 webserver.cc:460] Webserver started at http://127.22.144.130:35895/ using document root <none> and password file <none>
I20260811 04:42:22.257767 2225 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:22.257869 2225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:22.258140 2225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:22.260327 2225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "d0853473edc649b483c8887e464868c0"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "aa42962a16d698a96eafbb254963bd01"
server_key_iv: "28b5bb3ef2e3f8cf44485ba0a8a911e2"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.260957 2225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "d0853473edc649b483c8887e464868c0"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "aa42962a16d698a96eafbb254963bd01"
server_key_iv: "28b5bb3ef2e3f8cf44485ba0a8a911e2"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.265206 2225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:42:22.268006 2242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.269596 2225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:42:22.269762 2225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "d0853473edc649b483c8887e464868c0"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "aa42962a16d698a96eafbb254963bd01"
server_key_iv: "28b5bb3ef2e3f8cf44485ba0a8a911e2"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.269932 2225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:22.281103 2225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:22.284364 2225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:22.284587 2225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:22.285259 2225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:22.286188 2225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:22.286260 2225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.286329 2225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:22.286376 2225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.296800 2225 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:34133
I20260811 04:42:22.296823 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:34133 every 8 connection(s)
I20260811 04:42:22.298108 2225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:42:22.306280 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 2225
I20260811 04:42:22.306437 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:42:22.306892 23106 external_mini_cluster.cc:1428] Setting key 8068bc003cfcb2834485910f6349972b
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[1950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:42:22.313158 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.300025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:48797 (local address 127.22.144.190:36375)
0811 04:42:22.300199 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.300203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.301023 (+ 820us) server_negotiation.cc:407] Connection header received
0811 04:42:22.302685 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.302689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.302749 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.302840 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:22.304076 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.304643 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.305901 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.306166 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.306289 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.309422 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:22.309427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:22.309429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:22.309453 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:22.311014 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.311586 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.311591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.311593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.311679 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.312033 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.312039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.312043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.312235 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:22.312347 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.312867 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.312994 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 04:42:22.314059 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.300418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36375 (local address 127.22.144.130:48797)
0811 04:42:22.300870 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.300912 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.302364 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.303030 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.303037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.303419 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:22.303912 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.303927 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.304781 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.304785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.305775 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.305785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.306334 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.306338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.306570 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.307290 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:22.307307 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:22.309240 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:22.311172 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:22.311179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:22.311195 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:22.311470 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:22.311776 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:22.311780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:22.311782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:22.311933 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:22.312347 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:22.312355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:22.312621 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.312813 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.313164 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260811 04:42:22.315289 2377 heartbeater.cc:347] Connected to a master server at 127.22.144.190:36375
I20260811 04:42:22.315614 2377 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:22.316311 2377 heartbeater.cc:511] Master 127.22.144.190:36375 requested a full tablet report, sending...
I20260811 04:42:22.317747 2000 ts_manager.cc:194] Registered new tserver with Master: d0853473edc649b483c8887e464868c0 (127.22.144.130:34133)
I20260811 04:42:22.318457 2000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:48797
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:42:22.342785 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36375
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:42:22.462159 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:42:22.462510 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:42:22.462563 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:42:22.466732 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:42:22.466796 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:42:22.466857 2383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:42:22.472338 2383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:42:22.473464 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:42:22.474623 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:42:22.485199 2389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:22.485196 2392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:42:22.485231 2388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:42:22.485713 2383 server_base.cc:1034] running on GCE node
I20260811 04:42:22.486131 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:42:22.486786 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:42:22.487984 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786423342487951 us; error 51 us; skew 500 ppm
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[1950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:42:22.491313 2383 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:42:22.492448 2383 webserver.cc:460] Webserver started at http://127.22.144.131:38593/ using document root <none> and password file <none>
I20260811 04:42:22.493101 2383 fs_manager.cc:359] Metadata directory not provided
I20260811 04:42:22.493201 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:42:22.493471 2383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:42:22.495719 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "3072962ff1544fcea56caa3e1e35e07b"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "342fbcd46d80188ef46fac41989fedba"
server_key_iv: "59076408f7e126b28a3057ec012b651f"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.496306 2383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "3072962ff1544fcea56caa3e1e35e07b"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "342fbcd46d80188ef46fac41989fedba"
server_key_iv: "59076408f7e126b28a3057ec012b651f"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.500392 2383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 04:42:22.503271 2399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.504812 2383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 04:42:22.505002 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "3072962ff1544fcea56caa3e1e35e07b"
format_stamp: "Formatted at 2026-08-11 04:42:22 on dist-test-slave-ss4n"
server_key: "342fbcd46d80188ef46fac41989fedba"
server_key_iv: "59076408f7e126b28a3057ec012b651f"
server_key_version: "encryptionkey@0"
I20260811 04:42:22.505162 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:42:22.536710 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:42:22.540555 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:42:22.540820 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:42:22.541654 2383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:42:22.542752 2383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:42:22.542841 2383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.542928 2383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:42:22.542974 2383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:42:22.554708 2383 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:34039
I20260811 04:42:22.554725 2528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:34039 every 8 connection(s)
I20260811 04:42:22.555816 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:42:22.561784 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 2383
I20260811 04:42:22.561954 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:42:22.562325 23106 external_mini_cluster.cc:1428] Setting key 1e0596fe47aa32a4de45866bb2b5c790
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[1950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423342, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:42:22.571997 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.558045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:41167 (local address 127.22.144.190:36375)
0811 04:42:22.558237 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.558241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.559078 (+ 837us) server_negotiation.cc:407] Connection header received
0811 04:42:22.560630 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.560634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.560698 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.560810 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:22.562191 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.562936 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.564312 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.564555 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.564671 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.567533 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:22.567543 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:22.567545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:22.567574 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:22.569286 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.570084 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.570091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.570093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.570201 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.570643 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.570648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.570650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.570900 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:22.571035 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.571625 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.571773 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 04:42:22.572902 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.558413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36375 (local address 127.22.144.131:41167)
0811 04:42:22.558903 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.558950 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.560390 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.561000 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.561008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.561384 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:42:22.562011 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.562029 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.563072 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.563075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.564184 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.564195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.564713 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.564717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.564892 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.565647 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:42:22.565665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:42:22.567338 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:42:22.569574 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:22.569584 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:22.569606 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:22.569949 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:22.570325 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:42:22.570330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:42:22.570332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:42:22.570517 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:42:22.571034 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:42:22.571044 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:42:22.571333 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.571558 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.571956 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260811 04:42:22.574286 2529 heartbeater.cc:347] Connected to a master server at 127.22.144.190:36375
I20260811 04:42:22.574576 2529 heartbeater.cc:464] Registering TS with master...
I20260811 04:42:22.575129 2529 heartbeater.cc:511] Master 127.22.144.190:36375 requested a full tablet report, sending...
I20260811 04:42:22.576467 2000 ts_manager.cc:194] Registered new tserver with Master: 3072962ff1544fcea56caa3e1e35e07b (127.22.144.131:34039)
I20260811 04:42:22.577266 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:42:22.577343 2000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:41167
I20260811 04:42:22.588475 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.579614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.22.144.190:36375)
0811 04:42:22.579747 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.579750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.579932 (+ 182us) server_negotiation.cc:407] Connection header received
0811 04:42:22.580040 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.580042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.580107 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.580180 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:42:22.581184 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.581950 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.582977 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.583224 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.583320 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.584877 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:42:22.584880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:42:22.584883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:42:22.584909 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:42:22.586764 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.587215 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.587218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.587220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.587301 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:42:22.587633 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:42:22.587636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:42:22.587637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:42:22.587872 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:42:22.587991 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.588234 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.588341 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:42:22.592023 2000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:42:22.595610 2000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:42:22.611761 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.605335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43859 (local address 127.0.0.1:58006)
0811 04:42:22.605754 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.605815 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.605969 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.606881 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.606890 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.606919 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:42:22.607131 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.607139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.608698 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.608702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.610150 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.610160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.611162 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.611166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.611278 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.611323 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.611466 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.611593 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":47,"thread_start_us":87,"threads_started":1}
I20260811 04:42:22.611774 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.605601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.22.144.129:43859)
0811 04:42:22.606316 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.606324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.606352 (+ 28us) server_negotiation.cc:407] Connection header received
0811 04:42:22.606439 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.606445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.606680 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.607300 (+ 620us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:42:22.607432 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.608562 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.610277 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.611007 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.611147 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.611290 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.611501 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.611622 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":118,"threads_started":1}
I20260811 04:42:22.612128 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.605750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34133 (local address 127.0.0.1:38388)
0811 04:42:22.606898 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.606914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.607023 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.607602 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.607604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.607623 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:42:22.607725 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.607731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.609067 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.609071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.610575 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.610583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.611689 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.611692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.611779 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.611807 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.611881 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.611969 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":160,"spinlock_wait_cycles":1920,"thread_start_us":94,"threads_started":1}
I20260811 04:42:22.612388 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.606456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.22.144.131:34039)
0811 04:42:22.606881 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.606893 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.607131 (+ 238us) server_negotiation.cc:407] Connection header received
0811 04:42:22.607351 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.607357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.607506 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.607639 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:42:22.608079 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.609174 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.610696 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.611522 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.611641 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.611752 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.612116 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.612238 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":106,"threads_started":1}
I20260811 04:42:22.612731 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.606126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34039 (local address 127.0.0.1:37550)
0811 04:42:22.607028 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.607042 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.607201 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.607806 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.607810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.607830 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:42:22.607937 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.607943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.609325 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.609328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.610548 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.610560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.611825 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.611831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.611957 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.611985 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.612536 (+ 551us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.612614 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260811 04:42:22.612774 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.606201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.22.144.130:34133)
0811 04:42:22.606756 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.606768 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.607120 (+ 352us) server_negotiation.cc:407] Connection header received
0811 04:42:22.607218 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.607222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.607438 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.607611 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:42:22.607861 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.608786 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.610703 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.611388 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.611506 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.611626 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.612420 (+ 794us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.612595 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":138,"threads_started":1}
I20260811 04:42:22.615083 2145 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9173038c6a41f6a0670c2bdd9df4d4 (DEFAULT_TABLE table=test-table [id=58755692082d4261acf9f0c1756c9f15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:42:22.615586 2300 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9173038c6a41f6a0670c2bdd9df4d4 (DEFAULT_TABLE table=test-table [id=58755692082d4261acf9f0c1756c9f15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:42:22.615586 2455 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9173038c6a41f6a0670c2bdd9df4d4 (DEFAULT_TABLE table=test-table [id=58755692082d4261acf9f0c1756c9f15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:42:22.616150 2145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9173038c6a41f6a0670c2bdd9df4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:22.616642 2455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9173038c6a41f6a0670c2bdd9df4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:22.616636 2300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9173038c6a41f6a0670c2bdd9df4d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:42:22.622776 2547 tablet_bootstrap.cc:492] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: Bootstrap starting.
I20260811 04:42:22.624334 2549 tablet_bootstrap.cc:492] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Bootstrap starting.
I20260811 04:42:22.625295 2547 tablet_bootstrap.cc:654] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:22.626108 2547 log.cc:824] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:22.626536 2549 tablet_bootstrap.cc:654] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:22.626801 2550 tablet_bootstrap.cc:492] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Bootstrap starting.
I20260811 04:42:22.627295 2549 log.cc:824] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:22.629048 2549 tablet_bootstrap.cc:492] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: No bootstrap required, opened a new log
I20260811 04:42:22.629174 2547 tablet_bootstrap.cc:492] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: No bootstrap required, opened a new log
I20260811 04:42:22.629256 2549 ts_tablet_manager.cc:1397] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 04:42:22.629402 2547 ts_tablet_manager.cc:1397] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 04:42:22.630980 2550 tablet_bootstrap.cc:654] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Neither blocks nor log segments found. Creating new log.
I20260811 04:42:22.632298 2550 log.cc:824] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Log is configured to *not* fsync() on all Append() calls
I20260811 04:42:22.633014 2547 raft_consensus.cc:348] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.633373 2547 raft_consensus.cc:374] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:22.633458 2547 raft_consensus.cc:729] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 588d418310e74fd9a9dd124bc9c26f1a, State: Initialized, Role: FOLLOWER
I20260811 04:42:22.633365 2549 raft_consensus.cc:348] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.633762 2549 raft_consensus.cc:374] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:22.633832 2549 raft_consensus.cc:729] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0853473edc649b483c8887e464868c0, State: Initialized, Role: FOLLOWER
I20260811 04:42:22.633983 2547 consensus_queue.cc:260] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.634253 2549 consensus_queue.cc:260] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.634824 2220 heartbeater.cc:503] Master 127.22.144.190:36375 was elected leader, sending a full tablet report...
I20260811 04:42:22.635077 2547 ts_tablet_manager.cc:1428] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 04:42:22.635245 2549 ts_tablet_manager.cc:1428] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:42:22.635615 2377 heartbeater.cc:503] Master 127.22.144.190:36375 was elected leader, sending a full tablet report...
I20260811 04:42:22.636900 2550 tablet_bootstrap.cc:492] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: No bootstrap required, opened a new log
I20260811 04:42:22.637125 2550 ts_tablet_manager.cc:1397] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 04:42:22.640777 2550 raft_consensus.cc:348] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.640992 2550 raft_consensus.cc:374] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:42:22.641058 2550 raft_consensus.cc:729] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3072962ff1544fcea56caa3e1e35e07b, State: Initialized, Role: FOLLOWER
I20260811 04:42:22.641449 2550 consensus_queue.cc:260] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.642272 2529 heartbeater.cc:503] Master 127.22.144.190:36375 was elected leader, sending a full tablet report...
I20260811 04:42:22.642591 2550 ts_tablet_manager.cc:1428] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 04:42:22.720156 2554 raft_consensus.cc:482] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:42:22.720455 2554 raft_consensus.cc:504] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.721771 2554 leader_election.cc:288] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3072962ff1544fcea56caa3e1e35e07b (127.22.144.131:34039), d0853473edc649b483c8887e464868c0 (127.22.144.130:34133)
I20260811 04:42:22.726500 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.722342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:49177 (local address 127.22.144.130:34133)
0811 04:42:22.722473 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.722476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.722777 (+ 301us) server_negotiation.cc:407] Connection header received
0811 04:42:22.722867 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.722874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.722953 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.723054 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:42:22.723433 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.724058 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.725396 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.725867 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.725974 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.726037 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.726280 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.726362 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:42:22.726573 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.722100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34039 (local address 127.22.144.129:37993)
0811 04:42:22.722781 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.722800 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.722946 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.723296 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.723299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.723325 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:42:22.723556 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.723567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.724444 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.724446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.725224 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.725233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.726102 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.726104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.726185 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.726203 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.726303 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.726367 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":49,"thread_start_us":125,"threads_started":1}
I20260811 04:42:22.726570 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.722234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:37993 (local address 127.22.144.131:34039)
0811 04:42:22.722408 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.722413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.722922 (+ 509us) server_negotiation.cc:407] Connection header received
0811 04:42:22.723098 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.723102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.723159 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.723257 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:42:22.723694 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.724320 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.725363 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.725962 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.726092 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.726148 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.726337 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.726422 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 04:42:22.726594 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.722100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34133 (local address 127.22.144.129:49177)
0811 04:42:22.722644 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.722658 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:42:22.722774 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:42:22.723082 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:42:22.723085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:42:22.723104 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:42:22.723284 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.723293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.724176 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.724180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.725224 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:42:22.725235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.726000 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.726007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:42:22.726130 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.726157 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 04:42:22.726240 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 04:42:22.726336 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":3,"thread_start_us":156,"threads_started":1}
I20260811 04:42:22.727325 2477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9173038c6a41f6a0670c2bdd9df4d4" candidate_uuid: "588d418310e74fd9a9dd124bc9c26f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3072962ff1544fcea56caa3e1e35e07b" is_pre_election: true
I20260811 04:42:22.727391 2323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9173038c6a41f6a0670c2bdd9df4d4" candidate_uuid: "588d418310e74fd9a9dd124bc9c26f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0853473edc649b483c8887e464868c0" is_pre_election: true
I20260811 04:42:22.727667 2477 raft_consensus.cc:2454] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588d418310e74fd9a9dd124bc9c26f1a in term 0.
I20260811 04:42:22.727733 2323 raft_consensus.cc:2454] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 588d418310e74fd9a9dd124bc9c26f1a in term 0.
I20260811 04:42:22.728214 2094 leader_election.cc:302] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3072962ff1544fcea56caa3e1e35e07b, 588d418310e74fd9a9dd124bc9c26f1a, d0853473edc649b483c8887e464868c0; no voters:
I20260811 04:42:22.728750 2554 raft_consensus.cc:2781] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:42:22.728830 2554 raft_consensus.cc:482] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:42:22.728860 2554 raft_consensus.cc:3037] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:22.729895 2554 raft_consensus.cc:504] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.730206 2554 leader_election.cc:288] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [CANDIDATE]: Term 1 election: Requested vote from peers 3072962ff1544fcea56caa3e1e35e07b (127.22.144.131:34039), d0853473edc649b483c8887e464868c0 (127.22.144.130:34133)
I20260811 04:42:22.730628 2477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9173038c6a41f6a0670c2bdd9df4d4" candidate_uuid: "588d418310e74fd9a9dd124bc9c26f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3072962ff1544fcea56caa3e1e35e07b"
I20260811 04:42:22.730698 2323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9173038c6a41f6a0670c2bdd9df4d4" candidate_uuid: "588d418310e74fd9a9dd124bc9c26f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0853473edc649b483c8887e464868c0"
I20260811 04:42:22.730778 2477 raft_consensus.cc:3037] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:22.730818 2323 raft_consensus.cc:3037] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:42:22.732226 2477 raft_consensus.cc:2454] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588d418310e74fd9a9dd124bc9c26f1a in term 1.
I20260811 04:42:22.732226 2323 raft_consensus.cc:2454] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 588d418310e74fd9a9dd124bc9c26f1a in term 1.
I20260811 04:42:22.732553 2094 leader_election.cc:302] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3072962ff1544fcea56caa3e1e35e07b, 588d418310e74fd9a9dd124bc9c26f1a; no voters:
I20260811 04:42:22.732792 2554 raft_consensus.cc:2781] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:42:22.733033 2554 raft_consensus.cc:686] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [term 1 LEADER]: Becoming Leader. State: Replica: 588d418310e74fd9a9dd124bc9c26f1a, State: Running, Role: LEADER
I20260811 04:42:22.733400 2554 consensus_queue.cc:237] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } }
I20260811 04:42:22.736912 1998 catalog_manager.cc:5495] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 588d418310e74fd9a9dd124bc9c26f1a (127.22.144.129). New cstate: current_term: 1 leader_uuid: "588d418310e74fd9a9dd124bc9c26f1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "588d418310e74fd9a9dd124bc9c26f1a" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 } health_report { overall_health: UNKNOWN } } }
I20260811 04:42:22.768373 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.764313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.22.144.129:43859)
0811 04:42:22.764470 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 04:42:22.764475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:42:22.764492 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:42:22.764736 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:42:22.764739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:42:22.764801 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:42:22.764923 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:42:22.765209 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.765814 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.766722 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:42:22.766955 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:42:22.767077 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:42:22.767473 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:42:22.767936 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:42:22.768082 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 04:42:22.768159 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 04:42:22.768223 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 04:42:22.774802 2477 raft_consensus.cc:1261] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 1 FOLLOWER]: Refusing update from remote peer 588d418310e74fd9a9dd124bc9c26f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:42:22.774803 2323 raft_consensus.cc:1261] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 1 FOLLOWER]: Refusing update from remote peer 588d418310e74fd9a9dd124bc9c26f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:42:22.775648 2560 consensus_queue.cc:1035] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0853473edc649b483c8887e464868c0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:42:22.775890 2554 consensus_queue.cc:1035] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3072962ff1544fcea56caa3e1e35e07b" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:42:22.784636 2565 mvcc.cc:205] Tried to move back new op lower bound from 7317190011997487104 to 7317190011839262720. Current Snapshot: MvccSnapshot[applied={T|T < 7317190011997487104}]
I20260811 04:42:22.794168 1998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:42:22.794369 1998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:42:22.796893 1998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f8cf5cf8bd846729599ae66bdd22314: Sending DeleteTablet for 3 replicas of tablet 2c9173038c6a41f6a0670c2bdd9df4d4
I20260811 04:42:22.797850 2455 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c9173038c6a41f6a0670c2bdd9df4d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:42:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:37550
I20260811 04:42:22.797863 2145 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c9173038c6a41f6a0670c2bdd9df4d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:42:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:58006
I20260811 04:42:22.797865 2300 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c9173038c6a41f6a0670c2bdd9df4d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:42:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:38388
I20260811 04:42:22.798382 2573 tablet_replica.cc:330] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 588d418310e74fd9a9dd124bc9c26f1a: stopping tablet replica
I20260811 04:42:22.798714 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 2070
I20260811 04:42:22.801923 2572 tablet_replica.cc:330] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: stopping tablet replica
I20260811 04:42:22.802249 2574 tablet_replica.cc:330] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: stopping tablet replica
I20260811 04:42:22.802574 2574 raft_consensus.cc:2229] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:42:22.802662 2572 raft_consensus.cc:2229] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:42:22.802899 2574 raft_consensus.cc:2258] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:42:22.802994 2572 raft_consensus.cc:2258] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:42:22.804412 2574 ts_tablet_manager.cc:1905] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:42:22.804821 2572 ts_tablet_manager.cc:1905] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:42:22.806917 2574 ts_tablet_manager.cc:1918] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:42:22.807036 2574 log.cc:1196] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/2c9173038c6a41f6a0670c2bdd9df4d4
I20260811 04:42:22.807351 2574 ts_tablet_manager.cc:1939] T 2c9173038c6a41f6a0670c2bdd9df4d4 P 3072962ff1544fcea56caa3e1e35e07b: Deleting consensus metadata
I20260811 04:42:22.808251 2572 ts_tablet_manager.cc:1918] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:42:22.808358 2572 log.cc:1196] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/2c9173038c6a41f6a0670c2bdd9df4d4
I20260811 04:42:22.808611 1986 catalog_manager.cc:4841] TS 3072962ff1544fcea56caa3e1e35e07b (127.22.144.131:34039): tablet 2c9173038c6a41f6a0670c2bdd9df4d4 (table test-table [id=58755692082d4261acf9f0c1756c9f15]) successfully deleted
I20260811 04:42:22.808696 2572 ts_tablet_manager.cc:1939] T 2c9173038c6a41f6a0670c2bdd9df4d4 P d0853473edc649b483c8887e464868c0: Deleting consensus metadata
I20260811 04:42:22.810070 1985 catalog_manager.cc:4841] TS d0853473edc649b483c8887e464868c0 (127.22.144.130:34133): tablet 2c9173038c6a41f6a0670c2bdd9df4d4 (table test-table [id=58755692082d4261acf9f0c1756c9f15]) successfully deleted
I20260811 04:42:22.810910 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 2225
W20260811 04:42:22.810933 1986 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:43859 (error 108)
I20260811 04:42:22.812850 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:42:22.812471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43859 (local address 127.0.0.1:58012)
0811 04:42:22.812637 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 04:42:22.812737 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 04:42:22.813031 1986 catalog_manager.cc:4584] TS 588d418310e74fd9a9dd124bc9c26f1a (127.22.144.129:43859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c9173038c6a41f6a0670c2bdd9df4d4: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43859: connect: Connection refused (error 111)
I20260811 04:42:22.819343 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 2383
I20260811 04:42:22.829006 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 1966
2026-08-11T04:42:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2584 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[2578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[2578](info): set up 2 sockets
Aug 11 04:42:22 dist-test-slave-ss4n krb5kdc[2578](info): commencing operation
krb5kdc: starting...
W20260811 04:42:23.982566 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:42:24 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:42:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/rangeradmin_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/rangeradmin_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/rangerlookup_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/rangerlookup_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
I20260811 04:42:24.190312 23106 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZWTiLN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/postgres -l logfile start
2026-08-11 04:42:27.104 UTC [2612] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 04:42:27.105 UTC [2612] LOG: listening on IPv4 address "127.22.144.148", port 47809
2026-08-11 04:42:27.109 UTC [2612] LOG: listening on Unix socket "/tmp/.s.PGSQL.47809"
2026-08-11 04:42:27.133 UTC [2615] LOG: database system was shut down at 2026-08-11 04:42:25 UTC
2026-08-11 04:42:27.137 UTC [2612] LOG: database system is ready to accept connections
I20260811 04:42:28.198525 23106 mini_postgres.cc:96] Postgres bound to 47809
2026-08-11 04:42:28.207 UTC [2623] FATAL: database "slave" does not exist
127.22.144.148:47809 - accepting connections
I20260811 04:42:28.208154 23106 mini_ranger.cc:161] Starting Ranger...
I20260811 04:42:28.225385 23106 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 04:42:28.817764 23106 mini_ranger.cc:87] Created ranger Postgres database
I20260811 04:42:28.817880 23106 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-admin
2026-08-11 04:42:29,153 [I] DB FLAVOR :POSTGRES
2026-08-11 04:42:29,153 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 04:42:29,154 [I] Checking connection..
2026-08-11 04:42:29,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 04:42:29,424 [I] Checking connection passed.
2026-08-11 04:42:29,424 [I] --------- Verifying version history table ---------
2026-08-11 04:42:29,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 04:42:29,719 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 04:42:29,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 04:42:29,990 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 04:42:29,991 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 04:42:29,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 04:42:30.227 UTC [2698] WARNING: there is no transaction in progress
2026-08-11 04:42:30,236 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 04:42:30,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 04:42:30,484 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 04:42:30,484 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 04:42:30,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 04:42:30,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 04:42:31,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-11 04:42:31,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 04:42:31,538 [I] Table x_portal_user does not exist in database ranger
2026-08-11 04:42:31,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 04:42:31,801 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 04:42:31,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 04:42:32,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 04:42:32,307 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 04:42:32,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 04:42:33.418 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.438 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.457 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.474 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.495 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.516 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.584 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.593 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.610 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.623 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.635 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.652 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.664 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.672 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.681 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:33.690 UTC [2823] WARNING: there is no transaction in progress
2026-08-11 04:42:34,480 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 04:42:34,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 04:42:34,717 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 04:42:34,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 04:42:34,968 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 04:42:34,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 04:42:35,244 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 04:42:35,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 04:42:35,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 04:42:35,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-11 04:42:36,050 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 04:42:36,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 04:42:36,318 [I] DB_PATCHES have already been applied
I20260811 04:42:36.325217 23106 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZWTiLN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 04:42:36.325294 23106 mini_ranger.cc:193] Using host: 127.22.144.148
I20260811 04:42:36.330313 23106 mini_ranger.cc:227] Ranger admin URL: http://127.22.144.148:51083
Aug 11, 2026 4:42:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 4:42:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 4:42:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 4:42:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 4:42:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 4:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 4:42:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 04:42:37.648777 23106 mini_ranger.cc:160] Time spent starting Ranger: real 9.441s user 0.009s sys 0.005s
Aug 11, 2026 4:42:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 4:42:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 4:42:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 4:42:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 4:42:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 04:43:00.166085 23106 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/rangerkms_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/rangerkms_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.144.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.144.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 04:43:00 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423380, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 04:43:00.286597 23106 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 04:43:00.300129 23106 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 04:43:01.418993 23106 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 04:43:01.419104 23106 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms
I20260811 04:43:01.419221 23106 mini_ranger_kms.cc:227] Using postgres at 127.22.144.148:47809
2026-08-11 04:43:02,078 [I] DB FLAVOR :POSTGRES
2026-08-11 04:43:02,078 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 04:43:02,078 [I] Checking connection
2026-08-11 04:43:02,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 04:43:02,407 [I] connection success
2026-08-11 04:43:02,407 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 04:43:02,408 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 04:43:02,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 04:43:02,661 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 04:43:02,661 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 04:43:02,661 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 04:43:02,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:47809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 04:43:02,947 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 04:43:03.512065 23106 mini_ranger_kms.cc:310] Created kms service
I20260811 04:43:03.771553 23106 mini_ranger_kms.cc:326] Created kudu user
I20260811 04:43:03.818670 23106 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 04:43:03 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423380, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:04.267058 23106 mini_ranger_kms.cc:384] Added ranger policy
I20260811 04:43:04.267225 23106 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 04:43:04.267249 23106 mini_ranger_kms.cc:240] Using host: 127.22.144.148
I20260811 04:43:04.273144 23106 mini_ranger_kms.cc:276] Ranger KMS PID: 3029
I20260811 04:43:04.273298 23106 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.144.148:35553
Aug 11, 2026 4:43:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 4:43:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 4:43:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 4:43:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 4:43:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 4:43:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 04:43:06.706992 23106 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.420s user 0.000s sys 0.456s
I20260811 04:43:06.707152 23106 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 04:43:06.707284 23106 mini_ranger_kms.cc:401] 127.22.144.148:35553/kms/v1/keys
Aug 11 04:43:06 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786423386, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 04:43:07 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423386, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:43:07 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423386, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:43:10 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:43:10.094007 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35917
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:35917
--ranger_config_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.144.148:35553/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:43:10.244547 3087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:43:10.244851 3087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:43:10.244892 3087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:43:10.249168 3087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:43:10.249236 3087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:43:10.249257 3087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:43:10.249274 3087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:43:10.249289 3087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:43:10.255327 3087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.144.148:35553/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:35917
--ranger_config_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:43:10.256464 3087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:43:10.257474 3087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:43:10.268186 3093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:10.268224 3092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:10.268186 3095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:43:10.268283 3087 server_base.cc:1034] running on GCE node
I20260811 04:43:10.269433 3087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:43:10.270820 3087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:43:10.272009 3087 hybrid_clock.cc:648] HybridClock initialized: now 1786423390271992 us; error 32 us; skew 500 ppm
Aug 11 04:43:10 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423390, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:43:10.275341 3087 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:43:10.276296 3087 webserver.cc:460] Webserver started at http://127.22.144.190:42215/ using document root <none> and password file <none>
I20260811 04:43:10.276840 3087 fs_manager.cc:359] Metadata directory not provided
I20260811 04:43:10.276899 3087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:43:10.277091 3087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 04:43:10 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423390, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:10.339061 3087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "724fe7d5518d48fa8a9f56d724460add"
format_stamp: "Formatted at 2026-08-11 04:43:10 on dist-test-slave-ss4n"
server_key: "ff529d546a7131f780483982c1f79593"
server_key_iv: "898f281cbf2167e401f9554b3868a628"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:10.339861 3087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "724fe7d5518d48fa8a9f56d724460add"
format_stamp: "Formatted at 2026-08-11 04:43:10 on dist-test-slave-ss4n"
server_key: "ff529d546a7131f780483982c1f79593"
server_key_iv: "898f281cbf2167e401f9554b3868a628"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:10.345400 3087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 04:43:10.363737 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:10.365607 3087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:43:10.365763 3087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "724fe7d5518d48fa8a9f56d724460add"
format_stamp: "Formatted at 2026-08-11 04:43:10 on dist-test-slave-ss4n"
server_key: "ff529d546a7131f780483982c1f79593"
server_key_iv: "898f281cbf2167e401f9554b3868a628"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:10.365913 3087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:43:10.385259 3087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:43:10.388617 3087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:43:10.388896 3087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:43:10.398943 3087 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:35917
I20260811 04:43:10.398978 3161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:35917 every 8 connection(s)
I20260811 04:43:10.400045 3087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:43:10.404006 3162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:43:10.404934 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 3087
I20260811 04:43:10.405083 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:43:10.405416 23106 external_mini_cluster.cc:1428] Setting key d578b77e405b1bddaa6213a8ebddbfb9
I20260811 04:43:10.412175 3162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add: Bootstrap starting.
I20260811 04:43:10.415243 3162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add: Neither blocks nor log segments found. Creating new log.
Aug 11 04:43:10 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:43:10.416396 3162 log.cc:824] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add: Log is configured to *not* fsync() on all Append() calls
I20260811 04:43:10.418838 3162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add: No bootstrap required, opened a new log
I20260811 04:43:10.423136 3165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:10.406895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42162 (local address 127.22.144.190:35917)
0811 04:43:10.407471 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:10.407489 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:10.407722 (+ 233us) server_negotiation.cc:407] Connection header received
0811 04:43:10.408705 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:10.408726 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:10.409076 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:10.409423 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:43:10.410316 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:10.411533 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:10.412484 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:10.413011 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:10.413195 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:10.416301 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:43:10.416308 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:43:10.416323 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:43:10.416356 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:43:10.419300 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:10.420096 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:10.420103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:10.420110 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:10.420250 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:10.420617 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:10.420620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:10.420621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:10.421209 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:43:10.421412 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:10.421822 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 04:43:10.422129 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":249,"threads_started":1}
I20260811 04:43:10.423008 3162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724fe7d5518d48fa8a9f56d724460add" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35917 } }
I20260811 04:43:10.423276 3162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:43:10.423346 3162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724fe7d5518d48fa8a9f56d724460add, State: Initialized, Role: FOLLOWER
I20260811 04:43:10.423861 3162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724fe7d5518d48fa8a9f56d724460add" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35917 } }
I20260811 04:43:10.424034 3162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:43:10.424129 3162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:43:10.424268 3162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:43:10.425421 3162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724fe7d5518d48fa8a9f56d724460add" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35917 } }
I20260811 04:43:10.425835 3162 leader_election.cc:302] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 724fe7d5518d48fa8a9f56d724460add; no voters:
I20260811 04:43:10.426246 3162 leader_election.cc:288] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:43:10.426362 3168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:43:10.426581 3168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [term 1 LEADER]: Becoming Leader. State: Replica: 724fe7d5518d48fa8a9f56d724460add, State: Running, Role: LEADER
I20260811 04:43:10.426956 3168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724fe7d5518d48fa8a9f56d724460add" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35917 } }
I20260811 04:43:10.427640 3162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:43:10.428342 3168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [sys.catalog]: SysCatalogTable state changed. Reason: New leader 724fe7d5518d48fa8a9f56d724460add. Latest consensus state: current_term: 1 leader_uuid: "724fe7d5518d48fa8a9f56d724460add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724fe7d5518d48fa8a9f56d724460add" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35917 } } }
I20260811 04:43:10.428535 3168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [sys.catalog]: This master's current role is: LEADER
I20260811 04:43:10.429342 3169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "724fe7d5518d48fa8a9f56d724460add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724fe7d5518d48fa8a9f56d724460add" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35917 } } }
I20260811 04:43:10.429607 3176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:43:10.429682 3169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add [sys.catalog]: This master's current role is: LEADER
I20260811 04:43:10.433104 3176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:43:10.435642 3162 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 04:43:10.447448 3176 catalog_manager.cc:1347] Generated new cluster ID: cb5e9171f3904e938583012c44bd9846
I20260811 04:43:10.447548 3176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:43:10.470826 3176 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:43:10.472697 3176 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:43:10.485296 3176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add: Generated new TSK 0
I20260811 04:43:10.486483 3176 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:43:11.326352 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.144.148:35553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:35917
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:43:11.483736 3220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:43:11.484050 3220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:43:11.484179 3220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:43:11.488567 3220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:43:11.488660 3220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:43:11.488766 3220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:43:11.494560 3220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.144.148:35553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:43:11.495692 3220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:43:11.496739 3220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:43:11.509027 3229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:11.509509 3231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:11.509642 3228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:43:11.509950 3220 server_base.cc:1034] running on GCE node
I20260811 04:43:11.510456 3220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:43:11.511123 3220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:43:11.512319 3220 hybrid_clock.cc:648] HybridClock initialized: now 1786423391512278 us; error 53 us; skew 500 ppm
Aug 11 04:43:11 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:43:11.515625 3220 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:43:11.516678 3220 webserver.cc:460] Webserver started at http://127.22.144.129:40547/ using document root <none> and password file <none>
I20260811 04:43:11.517280 3220 fs_manager.cc:359] Metadata directory not provided
I20260811 04:43:11.517375 3220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:43:11.517714 3220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 04:43:11 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:11.533654 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "02215c31309c4a00bde2a2b8bad339b5"
format_stamp: "Formatted at 2026-08-11 04:43:11 on dist-test-slave-ss4n"
server_key: "d383cd919912d0f10ac9f5b5319da70b"
server_key_iv: "ee1d9fcaaf3da12217244cfcbe52253c"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:11.534794 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "02215c31309c4a00bde2a2b8bad339b5"
format_stamp: "Formatted at 2026-08-11 04:43:11 on dist-test-slave-ss4n"
server_key: "d383cd919912d0f10ac9f5b5319da70b"
server_key_iv: "ee1d9fcaaf3da12217244cfcbe52253c"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:11.540652 3220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 04:43:11.571725 3239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:11.581425 3220 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.005s sys 0.000s
I20260811 04:43:11.581650 3220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "02215c31309c4a00bde2a2b8bad339b5"
format_stamp: "Formatted at 2026-08-11 04:43:11 on dist-test-slave-ss4n"
server_key: "d383cd919912d0f10ac9f5b5319da70b"
server_key_iv: "ee1d9fcaaf3da12217244cfcbe52253c"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:11.581830 3220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:43:11.597685 3220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:43:11.603739 3220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:43:11.604063 3220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:43:11.604983 3220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:43:11.606319 3220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:43:11.606462 3220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:11.606611 3220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:43:11.606699 3220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:11.628284 3220 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:42833
I20260811 04:43:11.629695 3220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:43:11.629690 3375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:42833 every 8 connection(s)
I20260811 04:43:11.637773 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 3220
I20260811 04:43:11.637931 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:43:11.638271 23106 external_mini_cluster.cc:1428] Setting key f9a9e7bbb338fadb20e3df9f1bb78d21
Aug 11 04:43:11 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:43:11.651777 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:11.633025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:35917 (local address 127.22.144.129:53015)
0811 04:43:11.633957 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:11.634023 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:11.635247 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:11.635962 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:11.635972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:11.636538 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:43:11.637527 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:11.637549 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:11.639010 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:11.639014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:11.640697 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:11.640711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:11.641541 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:11.641548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:11.641841 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:11.642834 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:43:11.642859 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:43:11.645096 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:43:11.647871 (+ 2775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:43:11.647879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:43:11.647900 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:43:11.648316 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:43:11.648716 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:43:11.648720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:43:11.648722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:43:11.648924 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:43:11.649524 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:43:11.649532 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:43:11.649961 (+ 429us) client_negotiation.cc:769] Sending connection context
0811 04:43:11.650187 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 04:43:11.650580 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":184,"threads_started":1}
I20260811 04:43:11.652738 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:11.632491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53015 (local address 127.22.144.190:35917)
0811 04:43:11.634307 (+ 1816us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:11.634313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:11.634334 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:43:11.635545 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:11.635550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:11.635625 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:11.635738 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:43:11.637725 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:11.638605 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:11.640856 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:11.641207 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:11.641320 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:11.645321 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:43:11.645325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:43:11.645328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:43:11.645359 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:43:11.647696 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:11.648463 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:11.648467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:11.648470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:11.648581 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:11.649062 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:11.649066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:11.649068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:11.649324 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:43:11.649424 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:11.652385 (+ 2961us) server_negotiation.cc:299] Negotiation successful
0811 04:43:11.652551 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1728,"thread_start_us":122,"threads_started":1}
I20260811 04:43:11.654327 3377 heartbeater.cc:347] Connected to a master server at 127.22.144.190:35917
I20260811 04:43:11.654660 3377 heartbeater.cc:464] Registering TS with master...
I20260811 04:43:11.655402 3377 heartbeater.cc:511] Master 127.22.144.190:35917 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:43:11.657804 3121 ts_manager.cc:194] Registered new tserver with Master: 02215c31309c4a00bde2a2b8bad339b5 (127.22.144.129:42833)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:43:11.659585 3121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:53015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:43:11.682394 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.144.148:35553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:35917
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:43:11 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 04:43:11.865808 3383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:43:11.866297 3383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:43:11.866411 3383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:43:11.872846 3383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:43:11.873020 3383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:43:11.873162 3383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:43:11.881361 3383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.144.148:35553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:43:11.883069 3383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:43:11.884462 3383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:43:11.897867 3389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:11.897867 3390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:11.898561 3392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:43:11.899533 3383 server_base.cc:1034] running on GCE node
I20260811 04:43:11.900027 3383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:43:11.900817 3383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:43:11.902022 3383 hybrid_clock.cc:648] HybridClock initialized: now 1786423391901988 us; error 51 us; skew 500 ppm
Aug 11 04:43:11 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:43:11.906257 3383 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:43:11.907555 3383 webserver.cc:460] Webserver started at http://127.22.144.130:35771/ using document root <none> and password file <none>
I20260811 04:43:11.908581 3383 fs_manager.cc:359] Metadata directory not provided
I20260811 04:43:11.908703 3383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:43:11.909101 3383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 04:43:11 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:11.928635 3383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "c910d4ac221243e992f5654fe868a312"
format_stamp: "Formatted at 2026-08-11 04:43:11 on dist-test-slave-ss4n"
server_key: "eff35315c7b006f7c50089516bb03d19"
server_key_iv: "1b1988e3b600a751d3969c0fa821e75a"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:11.930025 3383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "c910d4ac221243e992f5654fe868a312"
format_stamp: "Formatted at 2026-08-11 04:43:11 on dist-test-slave-ss4n"
server_key: "eff35315c7b006f7c50089516bb03d19"
server_key_iv: "1b1988e3b600a751d3969c0fa821e75a"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:11.938535 3383 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.001s
I20260811 04:43:11.959064 3400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:11.962864 3383 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.001s
I20260811 04:43:11.963109 3383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "c910d4ac221243e992f5654fe868a312"
format_stamp: "Formatted at 2026-08-11 04:43:11 on dist-test-slave-ss4n"
server_key: "eff35315c7b006f7c50089516bb03d19"
server_key_iv: "1b1988e3b600a751d3969c0fa821e75a"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:11.963370 3383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:43:11.990501 3383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:43:11.994700 3383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:43:11.994921 3383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:43:11.995748 3383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:43:11.997017 3383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:43:11.997083 3383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:11.997138 3383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:43:11.997159 3383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:12.012516 3383 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:44791
I20260811 04:43:12.013813 3383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:43:12.015079 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 3383
I20260811 04:43:12.015232 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:43:12.015628 23106 external_mini_cluster.cc:1428] Setting key c5d9793fed9a2cddef2aa37b419a1733
I20260811 04:43:12.025882 3524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:44791 every 8 connection(s)
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Aug 11 04:43:12 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:43:12.060025 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.021778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:35917 (local address 127.22.144.130:59111)
0811 04:43:12.026464 (+ 4686us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.026529 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.030068 (+ 3539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.038321 (+ 8253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.038338 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.038916 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:43:12.039746 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.039773 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.041203 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.041207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.042787 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.042800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.043672 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.043697 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.044005 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.045069 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:43:12.045093 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:43:12.053277 (+ 8184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:43:12.055941 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:43:12.055951 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:43:12.055986 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:43:12.056419 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:43:12.056819 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:43:12.056822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:43:12.056824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:43:12.057214 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:43:12.057787 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:43:12.057795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:43:12.058197 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.058428 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.058797 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4344,"thread_start_us":150,"threads_started":1}
I20260811 04:43:12.060811 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.027050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:59111 (local address 127.22.144.190:35917)
0811 04:43:12.037679 (+ 10629us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.037685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.037709 (+ 24us) server_negotiation.cc:407] Connection header received
0811 04:43:12.037771 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.037775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.037835 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.037971 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:43:12.039990 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.040928 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.042953 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.043302 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.043393 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.053628 (+ 10235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:43:12.053633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:43:12.053637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:43:12.053680 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:43:12.055706 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:12.056567 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:12.056572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:12.056573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:12.056690 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:12.057337 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:12.057340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:12.057342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:12.057628 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:43:12.057726 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.060497 (+ 2771us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.060653 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10484}
I20260811 04:43:12.062762 3525 heartbeater.cc:347] Connected to a master server at 127.22.144.190:35917
I20260811 04:43:12.063112 3525 heartbeater.cc:464] Registering TS with master...
I20260811 04:43:12.063808 3525 heartbeater.cc:511] Master 127.22.144.190:35917 requested a full tablet report, sending...
I20260811 04:43:12.065297 3121 ts_manager.cc:194] Registered new tserver with Master: c910d4ac221243e992f5654fe868a312 (127.22.144.130:44791)
I20260811 04:43:12.066130 3121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:59111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:43:12.085672 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.144.148:35553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:35917
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:43:12 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
W20260811 04:43:12.284549 3535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:43:12.285005 3535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:43:12.285100 3535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:43:12.291239 3535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:43:12.291371 3535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:43:12.291487 3535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:43:12.300436 3535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.144.148:35553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:43:12.302009 3535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:43:12.303470 3535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:43:12.320502 3541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:12.320611 3542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:43:12.321190 3544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:43:12.325852 3535 server_base.cc:1034] running on GCE node
I20260811 04:43:12.326485 3535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:43:12.327370 3535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:43:12.334733 3535 hybrid_clock.cc:648] HybridClock initialized: now 1786423392334640 us; error 103 us; skew 500 ppm
Aug 11 04:43:12 dist-test-slave-ss4n krb5kdc[2578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423392, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:43:12.342523 3535 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:43:12.344072 3535 webserver.cc:460] Webserver started at http://127.22.144.131:35265/ using document root <none> and password file <none>
I20260811 04:43:12.345068 3535 fs_manager.cc:359] Metadata directory not provided
I20260811 04:43:12.345256 3535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:43:12.345889 3535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 04:43:12 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423392, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:12.379518 3535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "f9f89e0302034870a1d9654f1d785f81"
format_stamp: "Formatted at 2026-08-11 04:43:12 on dist-test-slave-ss4n"
server_key: "82213d4736a8c4f2f32f2ddfd6fa43bd"
server_key_iv: "c1b99ae8b3add649f4b9a3a5546b0996"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:12.385545 3535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "f9f89e0302034870a1d9654f1d785f81"
format_stamp: "Formatted at 2026-08-11 04:43:12 on dist-test-slave-ss4n"
server_key: "82213d4736a8c4f2f32f2ddfd6fa43bd"
server_key_iv: "c1b99ae8b3add649f4b9a3a5546b0996"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:12.396230 3535 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.001s sys 0.003s
I20260811 04:43:12.438382 3552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:12.440543 3535 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.004s sys 0.000s
I20260811 04:43:12.440707 3535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "f9f89e0302034870a1d9654f1d785f81"
format_stamp: "Formatted at 2026-08-11 04:43:12 on dist-test-slave-ss4n"
server_key: "82213d4736a8c4f2f32f2ddfd6fa43bd"
server_key_iv: "c1b99ae8b3add649f4b9a3a5546b0996"
server_key_version: "kuduclusterkey@0"
I20260811 04:43:12.440861 3535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:43:12.490406 3535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:43:12.494541 3535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:43:12.494818 3535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:43:12.495654 3535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:43:12.496940 3535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:43:12.497073 3535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:12.497188 3535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:43:12.497258 3535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:43:12.512678 3535 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:34819
I20260811 04:43:12.514029 3535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:43:12.520826 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 3535
I20260811 04:43:12.520962 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:43:12.521262 23106 external_mini_cluster.cc:1428] Setting key a80b176d1c82eed8d90507f5fcd06997
I20260811 04:43:12.523626 3683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:34819 every 8 connection(s)
Aug 11 04:43:12 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423392, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:43:12.544480 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.526272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:55929 (local address 127.22.144.190:35917)
0811 04:43:12.526396 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.526401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.528765 (+ 2364us) server_negotiation.cc:407] Connection header received
0811 04:43:12.530506 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.530510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.530570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.530650 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:43:12.532642 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.533512 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.535404 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.535775 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.535859 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.539676 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:43:12.539681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:43:12.539683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:43:12.539715 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:43:12.541544 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:12.542311 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:12.542316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:12.542318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:12.542418 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:12.542880 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:12.542883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:12.542885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:12.543159 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:43:12.543243 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.544160 (+ 917us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.544296 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 04:43:12.546591 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.526939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:35917 (local address 127.22.144.131:55929)
0811 04:43:12.528590 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.528655 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.530227 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.531119 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.531129 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.531601 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:43:12.532434 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.532458 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.533676 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.533680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.535244 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.535257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.536095 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.536099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.536381 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.537256 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:43:12.537279 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:43:12.539446 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:43:12.541682 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:43:12.541690 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:43:12.541710 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:43:12.542154 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:43:12.542575 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:43:12.542578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:43:12.542580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:43:12.542771 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:43:12.543336 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:43:12.543343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:43:12.543734 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.545000 (+ 1266us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.545387 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"thread_start_us":233,"threads_started":1}
I20260811 04:43:12.548532 3684 heartbeater.cc:347] Connected to a master server at 127.22.144.190:35917
I20260811 04:43:12.548800 3684 heartbeater.cc:464] Registering TS with master...
I20260811 04:43:12.549428 3684 heartbeater.cc:511] Master 127.22.144.190:35917 requested a full tablet report, sending...
I20260811 04:43:12.550770 3121 ts_manager.cc:194] Registered new tserver with Master: f9f89e0302034870a1d9654f1d785f81 (127.22.144.131:34819)
I20260811 04:43:12.551561 3121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:55929
I20260811 04:43:12.559710 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:43:12.574370 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.562661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.22.144.190:35917)
0811 04:43:12.562803 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.562808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.563583 (+ 775us) server_negotiation.cc:407] Connection header received
0811 04:43:12.563809 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.563813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.563889 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.563956 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:43:12.565249 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.566186 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.567450 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.567789 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.567880 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.569993 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:43:12.569997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:43:12.570000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:43:12.570032 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:43:12.572016 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:12.572867 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:12.572871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:12.572873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:12.572991 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:43:12.573313 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:43:12.573317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:43:12.573318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:43:12.573588 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:43:12.573706 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.574067 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.574204 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:43:12.578694 3121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:43:12.583484 3121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:43:12.624475 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.599697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34819 (local address 127.0.0.1:44090)
0811 04:43:12.605001 (+ 5304us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.605014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.605106 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.607671 (+ 2565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.607678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.607703 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:43:12.607859 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.607866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.609400 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.609404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.619453 (+ 10049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.619462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.621959 (+ 2497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.621963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.622095 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.622143 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.624143 (+ 2000us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.624310 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5233,"thread_start_us":102,"threads_started":1}
I20260811 04:43:12.626536 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.600218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.22.144.131:34819)
0811 04:43:12.604507 (+ 4289us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.604519 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.605203 (+ 684us) server_negotiation.cc:407] Connection header received
0811 04:43:12.605288 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.605293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.605559 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.605719 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:43:12.608021 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.609254 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.619646 (+ 10392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.620570 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.620721 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.620887 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.626241 (+ 5354us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.626382 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4173,"thread_start_us":165,"threads_started":1}
I20260811 04:43:12.626106 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.601931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:44791 (local address 127.0.0.1:33766)
0811 04:43:12.605895 (+ 3964us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.605909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.606002 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.608708 (+ 2706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.608712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.608732 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:43:12.608860 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.608866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.613027 (+ 4161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.613030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.614451 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.614463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.625643 (+ 11180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.625648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.625765 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.625793 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.625873 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.625953 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3905,"thread_start_us":161,"threads_started":1}
I20260811 04:43:12.632588 3602 tablet_service.cc:1467] Processing CreateTablet for tablet 0890f408720d452dae87d405de43f3f0 (DEFAULT_TABLE table=test-table [id=783048550fa144469aa48c350a64b023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:43:12.634178 3602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0890f408720d452dae87d405de43f3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:43:12.642007 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.602420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.22.144.130:44791)
0811 04:43:12.604860 (+ 2440us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.604867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.606324 (+ 1457us) server_negotiation.cc:407] Connection header received
0811 04:43:12.606407 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.606413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.606612 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.606794 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:43:12.611723 (+ 4929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.612873 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.620968 (+ 8095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.625237 (+ 4269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.625371 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.625557 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.641684 (+ 16127us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.641845 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2340,"thread_start_us":68,"threads_started":1}
I20260811 04:43:12.646230 3449 tablet_service.cc:1467] Processing CreateTablet for tablet 0890f408720d452dae87d405de43f3f0 (DEFAULT_TABLE table=test-table [id=783048550fa144469aa48c350a64b023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:43:12.649210 3449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0890f408720d452dae87d405de43f3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:43:12.655489 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.596502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42833 (local address 127.0.0.1:51248)
0811 04:43:12.603267 (+ 6765us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.603333 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.603473 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.616253 (+ 12780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.616256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.616277 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:43:12.616466 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.616475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.618003 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.618007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.619183 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.619195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.654879 (+ 35684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.654885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.655046 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.655088 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.655216 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.655325 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6530,"thread_start_us":601,"threads_started":1}
I20260811 04:43:12.655546 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.598075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.22.144.129:42833)
0811 04:43:12.614792 (+ 16717us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.614800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.614828 (+ 28us) server_negotiation.cc:407] Connection header received
0811 04:43:12.614912 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.614918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.615154 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.615351 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:43:12.616701 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.617850 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.627519 (+ 9669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.642812 (+ 15293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.642971 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.643136 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.655274 (+ 12138us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.655406 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16586,"thread_start_us":79,"threads_started":1}
I20260811 04:43:12.662703 3377 heartbeater.cc:503] Master 127.22.144.190:35917 was elected leader, sending a full tablet report...
I20260811 04:43:12.667268 3299 tablet_service.cc:1467] Processing CreateTablet for tablet 0890f408720d452dae87d405de43f3f0 (DEFAULT_TABLE table=test-table [id=783048550fa144469aa48c350a64b023]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:43:12.669137 3299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0890f408720d452dae87d405de43f3f0. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 04:43:12 dist-test-slave-ss4n krb5kdc[2578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423391, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:12.686038 3709 tablet_bootstrap.cc:492] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Bootstrap starting.
I20260811 04:43:12.689772 3709 tablet_bootstrap.cc:654] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Neither blocks nor log segments found. Creating new log.
I20260811 04:43:12.690248 3708 tablet_bootstrap.cc:492] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Bootstrap starting.
I20260811 04:43:12.695253 3709 log.cc:824] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Log is configured to *not* fsync() on all Append() calls
I20260811 04:43:12.697149 3708 tablet_bootstrap.cc:654] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Neither blocks nor log segments found. Creating new log.
I20260811 04:43:12.697865 3710 tablet_bootstrap.cc:492] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: Bootstrap starting.
I20260811 04:43:12.699276 3708 log.cc:824] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Log is configured to *not* fsync() on all Append() calls
I20260811 04:43:12.703119 3710 tablet_bootstrap.cc:654] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: Neither blocks nor log segments found. Creating new log.
I20260811 04:43:12.705528 3710 log.cc:824] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: Log is configured to *not* fsync() on all Append() calls
I20260811 04:43:12.709033 3709 tablet_bootstrap.cc:492] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: No bootstrap required, opened a new log
I20260811 04:43:12.709378 3709 ts_tablet_manager.cc:1397] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.007s
I20260811 04:43:12.716282 3708 tablet_bootstrap.cc:492] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: No bootstrap required, opened a new log
I20260811 04:43:12.716547 3708 ts_tablet_manager.cc:1397] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.000s
I20260811 04:43:12.716172 3709 raft_consensus.cc:348] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.716845 3709 raft_consensus.cc:374] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:43:12.716969 3709 raft_consensus.cc:729] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c910d4ac221243e992f5654fe868a312, State: Initialized, Role: FOLLOWER
I20260811 04:43:12.717657 3709 consensus_queue.cc:260] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.719318 3709 ts_tablet_manager.cc:1428] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260811 04:43:12.719919 3525 heartbeater.cc:503] Master 127.22.144.190:35917 was elected leader, sending a full tablet report...
I20260811 04:43:12.723764 3710 tablet_bootstrap.cc:492] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: No bootstrap required, opened a new log
I20260811 04:43:12.724056 3710 ts_tablet_manager.cc:1397] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.000s
I20260811 04:43:12.724845 3708 raft_consensus.cc:348] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.725169 3708 raft_consensus.cc:374] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:43:12.725227 3708 raft_consensus.cc:729] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f89e0302034870a1d9654f1d785f81, State: Initialized, Role: FOLLOWER
I20260811 04:43:12.728840 3708 consensus_queue.cc:260] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.729815 3684 heartbeater.cc:503] Master 127.22.144.190:35917 was elected leader, sending a full tablet report...
I20260811 04:43:12.730525 3710 raft_consensus.cc:348] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.730832 3710 raft_consensus.cc:374] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:43:12.730913 3710 raft_consensus.cc:729] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02215c31309c4a00bde2a2b8bad339b5, State: Initialized, Role: FOLLOWER
I20260811 04:43:12.731495 3710 consensus_queue.cc:260] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.732911 3710 ts_tablet_manager.cc:1428] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260811 04:43:12.732963 3708 ts_tablet_manager.cc:1428] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Time spent starting tablet: real 0.016s user 0.010s sys 0.000s
W20260811 04:43:12.771231 3526 tablet.cc:2369] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:43:12.776269 3685 tablet.cc:2369] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:43:12.857640 3715 raft_consensus.cc:482] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:43:12.857903 3715 raft_consensus.cc:504] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.859344 3715 leader_election.cc:288] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02215c31309c4a00bde2a2b8bad339b5 (127.22.144.129:42833), f9f89e0302034870a1d9654f1d785f81 (127.22.144.131:34819)
I20260811 04:43:12.872124 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.860747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:49069 (local address 127.22.144.129:42833)
0811 04:43:12.860942 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.860950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.861742 (+ 792us) server_negotiation.cc:407] Connection header received
0811 04:43:12.862370 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.862374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.862437 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.862512 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:43:12.864764 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.865671 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.869873 (+ 4202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.870555 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.870680 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.870742 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.871861 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.871970 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 04:43:12.873054 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.859813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42833 (local address 127.22.144.130:49069)
0811 04:43:12.861626 (+ 1813us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.861645 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.862193 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.864300 (+ 2107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.864304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.864344 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:43:12.864610 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.864619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.866222 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.866226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.867979 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.867993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.871222 (+ 3229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.871226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.871343 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.871701 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.872760 (+ 1059us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.872864 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1726,"mutex_wait_us":295,"thread_start_us":125,"threads_started":1}
I20260811 04:43:12.873848 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.860905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:57569 (local address 127.22.144.131:34819)
0811 04:43:12.861060 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.861065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.865768 (+ 4703us) server_negotiation.cc:407] Connection header received
0811 04:43:12.866017 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.866021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.866082 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.866142 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:43:12.866642 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.867489 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.870241 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.870912 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.871043 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.871098 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.873612 (+ 2514us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.873716 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 04:43:12.874372 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.860254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34819 (local address 127.22.144.130:57569)
0811 04:43:12.865658 (+ 5404us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:12.865675 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:43:12.865853 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:43:12.866310 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:43:12.866313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:43:12.866332 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:43:12.866483 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.866491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.867689 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.867693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.869028 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:43:12.869042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.871273 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.871278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:43:12.871412 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.871438 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 04:43:12.874099 (+ 2661us) client_negotiation.cc:240] Negotiation successful
0811 04:43:12.874196 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5326,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260811 04:43:12.876919 3323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890f408720d452dae87d405de43f3f0" candidate_uuid: "c910d4ac221243e992f5654fe868a312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02215c31309c4a00bde2a2b8bad339b5" is_pre_election: true
I20260811 04:43:12.877226 3608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890f408720d452dae87d405de43f3f0" candidate_uuid: "c910d4ac221243e992f5654fe868a312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f89e0302034870a1d9654f1d785f81" is_pre_election: true
I20260811 04:43:12.877346 3323 raft_consensus.cc:2454] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c910d4ac221243e992f5654fe868a312 in term 0.
I20260811 04:43:12.877616 3608 raft_consensus.cc:2454] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c910d4ac221243e992f5654fe868a312 in term 0.
I20260811 04:43:12.878219 3401 leader_election.cc:302] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02215c31309c4a00bde2a2b8bad339b5, c910d4ac221243e992f5654fe868a312; no voters:
I20260811 04:43:12.878611 3715 raft_consensus.cc:2781] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:43:12.878729 3715 raft_consensus.cc:482] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:43:12.878793 3715 raft_consensus.cc:3037] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:43:12.880594 3715 raft_consensus.cc:504] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.881043 3715 leader_election.cc:288] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [CANDIDATE]: Term 1 election: Requested vote from peers 02215c31309c4a00bde2a2b8bad339b5 (127.22.144.129:42833), f9f89e0302034870a1d9654f1d785f81 (127.22.144.131:34819)
I20260811 04:43:12.882437 3323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890f408720d452dae87d405de43f3f0" candidate_uuid: "c910d4ac221243e992f5654fe868a312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02215c31309c4a00bde2a2b8bad339b5"
I20260811 04:43:12.882757 3323 raft_consensus.cc:3037] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 0 FOLLOWER]: Advancing to term 1
W20260811 04:43:12.883571 3378 tablet.cc:2369] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:43:12.884259 3323 raft_consensus.cc:2454] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c910d4ac221243e992f5654fe868a312 in term 1.
I20260811 04:43:12.884747 3401 leader_election.cc:302] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02215c31309c4a00bde2a2b8bad339b5, c910d4ac221243e992f5654fe868a312; no voters:
I20260811 04:43:12.885131 3608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890f408720d452dae87d405de43f3f0" candidate_uuid: "c910d4ac221243e992f5654fe868a312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f89e0302034870a1d9654f1d785f81"
I20260811 04:43:12.885301 3608 raft_consensus.cc:3037] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:43:12.886996 3608 raft_consensus.cc:2454] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c910d4ac221243e992f5654fe868a312 in term 1.
I20260811 04:43:12.887741 3715 raft_consensus.cc:2781] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:43:12.888026 3715 raft_consensus.cc:686] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 1 LEADER]: Becoming Leader. State: Replica: c910d4ac221243e992f5654fe868a312, State: Running, Role: LEADER
I20260811 04:43:12.888437 3715 consensus_queue.cc:237] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } }
I20260811 04:43:12.893955 3119 catalog_manager.cc:5495] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 reported cstate change: term changed from 0 to 1, leader changed from <none> to c910d4ac221243e992f5654fe868a312 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "c910d4ac221243e992f5654fe868a312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c910d4ac221243e992f5654fe868a312" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 44791 } health_report { overall_health: HEALTHY } } }
I20260811 04:43:12.973721 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:12.967601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33774 (local address 127.22.144.130:44791)
0811 04:43:12.967732 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 04:43:12.967737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:43:12.967754 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:43:12.967898 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:43:12.967901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:43:12.967982 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:43:12.968112 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:43:12.968544 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.969471 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.971039 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:43:12.971425 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:43:12.971541 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:43:12.972053 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:43:12.972688 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:43:12.972828 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 04:43:12.973459 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 04:43:12.973557 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:43:12.981911 3608 raft_consensus.cc:1261] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 1 FOLLOWER]: Refusing update from remote peer c910d4ac221243e992f5654fe868a312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:43:12.982831 3715 consensus_queue.cc:1035] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9f89e0302034870a1d9654f1d785f81" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:43:12.990496 3323 raft_consensus.cc:1261] T 0890f408720d452dae87d405de43f3f0 P 02215c31309c4a00bde2a2b8bad339b5 [term 1 FOLLOWER]: Refusing update from remote peer c910d4ac221243e992f5654fe868a312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:43:12.991324 3715 consensus_queue.cc:1035] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02215c31309c4a00bde2a2b8bad339b5" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 42833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:43:13.012790 3726 mvcc.cc:205] Tried to move back new op lower bound from 7317190217643450368 to 7317190217280819200. Current Snapshot: MvccSnapshot[applied={T|T < 7317190217643450368}]
I20260811 04:43:13.020052 3730 mvcc.cc:205] Tried to move back new op lower bound from 7317190217643450368 to 7317190217280819200. Current Snapshot: MvccSnapshot[applied={T|T < 7317190217643450368}]
I20260811 04:43:13.020521 3727 mvcc.cc:205] Tried to move back new op lower bound from 7317190217643450368 to 7317190217280819200. Current Snapshot: MvccSnapshot[applied={T|T < 7317190217643450368}]
I20260811 04:43:13.036367 3119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:43:13.036600 3119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:43:13.040494 3119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 724fe7d5518d48fa8a9f56d724460add: Sending DeleteTablet for 3 replicas of tablet 0890f408720d452dae87d405de43f3f0
I20260811 04:43:13.041783 3299 tablet_service.cc:1514] Processing DeleteTablet for tablet 0890f408720d452dae87d405de43f3f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:43:13 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51248
I20260811 04:43:13.042538 3602 tablet_service.cc:1514] Processing DeleteTablet for tablet 0890f408720d452dae87d405de43f3f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:43:13 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:44090
I20260811 04:43:13.043963 3449 tablet_service.cc:1514] Processing DeleteTablet for tablet 0890f408720d452dae87d405de43f3f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:43:13 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:33766
I20260811 04:43:13.045277 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 3220
I20260811 04:43:13.051515 3742 tablet_replica.cc:330] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: stopping tablet replica
I20260811 04:43:13.052023 3742 raft_consensus.cc:2229] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:43:13.052399 3740 tablet_replica.cc:330] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: stopping tablet replica
I20260811 04:43:13.052793 3740 raft_consensus.cc:2229] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:43:13.053388 3742 raft_consensus.cc:2258] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:43:13.053897 3740 raft_consensus.cc:2258] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:43:13.064041 3742 ts_tablet_manager.cc:1905] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:43:13.067764 3742 ts_tablet_manager.cc:1918] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:43:13.070602 3740 ts_tablet_manager.cc:1905] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:43:13.071475 3742 log.cc:1196] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/0890f408720d452dae87d405de43f3f0
I20260811 04:43:13.072815 3742 ts_tablet_manager.cc:1939] T 0890f408720d452dae87d405de43f3f0 P c910d4ac221243e992f5654fe868a312: Deleting consensus metadata
I20260811 04:43:13.074064 3740 ts_tablet_manager.cc:1918] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:43:13.074188 3740 log.cc:1196] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/0890f408720d452dae87d405de43f3f0
I20260811 04:43:13.074540 3740 ts_tablet_manager.cc:1939] T 0890f408720d452dae87d405de43f3f0 P f9f89e0302034870a1d9654f1d785f81: Deleting consensus metadata
I20260811 04:43:13.076380 3105 catalog_manager.cc:4841] TS c910d4ac221243e992f5654fe868a312 (127.22.144.130:44791): tablet 0890f408720d452dae87d405de43f3f0 (table test-table [id=783048550fa144469aa48c350a64b023]) successfully deleted
I20260811 04:43:13.076839 3104 catalog_manager.cc:4841] TS f9f89e0302034870a1d9654f1d785f81 (127.22.144.131:34819): tablet 0890f408720d452dae87d405de43f3f0 (table test-table [id=783048550fa144469aa48c350a64b023]) successfully deleted
W20260811 04:43:13.077404 3103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:42833 (error 108)
I20260811 04:43:13.078507 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 3383
I20260811 04:43:13.091804 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:43:13.091357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:42833 (local address 127.0.0.1:51260)
0811 04:43:13.091600 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 04:43:13.091695 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:42833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260811 04:43:13.092243 3103 catalog_manager.cc:4584] TS 02215c31309c4a00bde2a2b8bad339b5 (127.22.144.129:42833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0890f408720d452dae87d405de43f3f0: Network error: Client connection negotiation failed: client connection to 127.22.144.129:42833: connect: Connection refused (error 111)
I20260811 04:43:13.094444 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 3535
I20260811 04:43:13.105967 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 3087
I20260811 04:43:13.115830 23106 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 04:43:13.462268 23106 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 04:43:13.462404 23106 mini_ranger.cc:66] Stopping Ranger...
I20260811 04:43:13.807852 23106 mini_ranger.cc:68] Stopped Ranger
2026-08-11 04:43:13.808 UTC [2612] LOG: received smart shutdown request
2026-08-11 04:43:13.812 UTC [2612] LOG: background worker "logical replication launcher" (PID 2621) exited with exit code 1
2026-08-11 04:43:13.813 UTC [2616] LOG: shutting down
2026-08-11 04:43:14.024 UTC [2612] LOG: database system is shut down
2026-08-11T04:43:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51317 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:43:14 dist-test-slave-ss4n krb5kdc[3758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:43:14 dist-test-slave-ss4n krb5kdc[3758](info): set up 2 sockets
Aug 11 04:43:14 dist-test-slave-ss4n krb5kdc[3758](info): commencing operation
krb5kdc: starting...
W20260811 04:43:15.335016 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:43:15 dist-test-slave-ss4n krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:43:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/rangeradmin_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/rangeradmin_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/rangerlookup_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/rangerlookup_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
I20260811 04:43:15.530181 23106 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZWTiLN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/postgres -l logfile start
2026-08-11 04:43:18.362 UTC [3792] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 04:43:18.362 UTC [3792] LOG: listening on IPv4 address "127.22.144.148", port 58121
2026-08-11 04:43:18.366 UTC [3792] LOG: listening on Unix socket "/tmp/.s.PGSQL.58121"
2026-08-11 04:43:18.389 UTC [3795] LOG: database system was shut down at 2026-08-11 04:43:16 UTC
2026-08-11 04:43:18.393 UTC [3792] LOG: database system is ready to accept connections
I20260811 04:43:19.427500 23106 mini_postgres.cc:96] Postgres bound to 58121
2026-08-11 04:43:19.435 UTC [3805] FATAL: database "slave" does not exist
127.22.144.148:58121 - accepting connections
I20260811 04:43:19.435760 23106 mini_ranger.cc:161] Starting Ranger...
I20260811 04:43:19.452046 23106 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 04:43:20.026793 23106 mini_ranger.cc:87] Created ranger Postgres database
I20260811 04:43:20.026890 23106 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-admin
2026-08-11 04:43:20,300 [I] DB FLAVOR :POSTGRES
2026-08-11 04:43:20,301 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 04:43:20,301 [I] Checking connection..
2026-08-11 04:43:20,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 04:43:20,602 [I] Checking connection passed.
2026-08-11 04:43:20,602 [I] --------- Verifying version history table ---------
2026-08-11 04:43:20,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 04:43:20,863 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 04:43:20,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 04:43:21,138 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 04:43:21,139 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 04:43:21,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 04:43:21.401 UTC [3879] WARNING: there is no transaction in progress
2026-08-11 04:43:21,409 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 04:43:21,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 04:43:21,684 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 04:43:21,685 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 04:43:21,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 04:43:21,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 04:43:22,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-11 04:43:22,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 04:43:22,737 [I] Table x_portal_user does not exist in database ranger
2026-08-11 04:43:22,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 04:43:23,017 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 04:43:23,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 04:43:23,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 04:43:23,510 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 04:43:23,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 04:43:24.600 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.619 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.639 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.658 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.677 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.698 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.773 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.781 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.797 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.808 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.819 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.834 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.845 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.853 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.861 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:24.869 UTC [3998] WARNING: there is no transaction in progress
2026-08-11 04:43:25,613 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 04:43:25,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 04:43:25,865 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 04:43:25,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 04:43:26,110 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 04:43:26,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 04:43:26,373 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 04:43:26,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 04:43:26,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 04:43:26,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-11 04:43:27,120 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 04:43:27,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 04:43:27,364 [I] DB_PATCHES have already been applied
I20260811 04:43:27.370705 23106 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZWTiLN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 04:43:27.370776 23106 mini_ranger.cc:193] Using host: 127.22.144.148
I20260811 04:43:27.375370 23106 mini_ranger.cc:227] Ranger admin URL: http://127.22.144.148:39009
Aug 11, 2026 4:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 4:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 4:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 4:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 4:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 4:43:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 4:43:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 04:43:28.598224 23106 mini_ranger.cc:160] Time spent starting Ranger: real 9.162s user 0.002s sys 0.011s
Aug 11, 2026 4:43:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 4:43:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 4:43:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 4:43:50 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 4:43:51 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 04:43:51.671504 23106 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.144.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.144.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/rangerkms_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.144.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/rangerkms_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.144.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.144.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.144.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/krb5kdc/HTTP_127.22.144.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 04:43:51 dist-test-slave-ss4n krb5kdc[3758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423431, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 04:43:51.788975 23106 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 04:43:51.800750 23106 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 04:43:52.859988 23106 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 04:43:52.860082 23106 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms
I20260811 04:43:52.860106 23106 mini_ranger_kms.cc:227] Using postgres at 127.22.144.148:58121
2026-08-11 04:43:53,038 [I] DB FLAVOR :POSTGRES
2026-08-11 04:43:53,039 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 04:43:53,039 [I] Checking connection
2026-08-11 04:43:53,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 04:43:53,310 [I] connection success
2026-08-11 04:43:53,310 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 04:43:53,310 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 04:43:53,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 04:43:53,584 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 04:43:53,585 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 04:43:53,585 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 04:43:53,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.144.148:58121/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 04:43:53,862 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 04:43:54.398051 23106 mini_ranger_kms.cc:310] Created kms service
I20260811 04:43:54.586221 23106 mini_ranger_kms.cc:326] Created kudu user
I20260811 04:43:54.634297 23106 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 04:43:54 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423431, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:43:55.105266 23106 mini_ranger_kms.cc:384] Added ranger policy
I20260811 04:43:55.105418 23106 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZWTiLN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZWTiLN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 04:43:55.105438 23106 mini_ranger_kms.cc:240] Using host: 127.22.144.148
I20260811 04:43:55.109943 23106 mini_ranger_kms.cc:276] Ranger KMS PID: 4221
I20260811 04:43:55.110033 23106 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.144.148:33349
Aug 11, 2026 4:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 4:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 4:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 4:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 4:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 4:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 04:43:57 dist-test-slave-ss4n krb5kdc[3758](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 04:43:57 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:43:58 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:44:28 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:44:28 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:44:58 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:44:58 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:45:28 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
Aug 11 04:45:28 dist-test-slave-ss4n krb5kdc[3758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423437, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.144.148@KRBTEST.COM for HTTP/127.22.144.148@KRBTEST.COM
I20260811 04:45:55.599882 23106 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 123.811s user 0.043s sys 0.310s
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:1024: Failure
Failed
Bad status: Timed out: Failed to start the Ranger KMS service: process with PID 4221 is not yet bound to any port at the specified addresses; last attempt running lsof returned 'Not found: could not find pattern of a bound port in lsof output: p4221
f196
n127.0.0.1:49926->127.22.144.148:58121'
I20260811 04:45:55.600997 23106 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 04:45:55.631886 23106 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 04:45:55.632047 23106 mini_ranger.cc:66] Stopping Ranger...
I20260811 04:45:55.987977 23106 mini_ranger.cc:68] Stopped Ranger
2026-08-11 04:45:55.988 UTC [3792] LOG: received smart shutdown request
2026-08-11 04:45:55.992 UTC [3792] LOG: background worker "logical replication launcher" (PID 3803) exited with exit code 1
2026-08-11 04:45:55.992 UTC [3796] LOG: shutting down
2026-08-11 04:45:56.159 UTC [3792] LOG: database system is shut down
2026-08-11T04:45:56Z chronyd exiting
I20260811 04:45:56.173177 23106 test_util.cc:175] -----------------------------------------------
I20260811 04:45:56.173246 23106 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786423274535345-23106-0
[ FAILED ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (162001 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:45:56 dist-test-slave-ss4n krb5kdc[4367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:45:56 dist-test-slave-ss4n krb5kdc[4367](info): set up 2 sockets
Aug 11 04:45:56 dist-test-slave-ss4n krb5kdc[4367](info): commencing operation
krb5kdc: starting...
W20260811 04:45:57.338249 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:45:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:45:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:45:57.484263 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:36171
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:36171,127.22.144.189:36819,127.22.144.188:41375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:45:57.607003 4385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:45:57.607352 4385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:45:57.607435 4385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:45:57.611568 4385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:45:57.611678 4385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:45:57.611720 4385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:45:57.611759 4385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:45:57.611801 4385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:45:57.617389 4385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:36171,127.22.144.189:36819,127.22.144.188:41375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:36171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:45:57.618682 4385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:45:57.619801 4385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:45:57.629208 4390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:45:57.629208 4391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:45:57.629236 4393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:45:57.629757 4385 server_base.cc:1034] running on GCE node
I20260811 04:45:57.630424 4385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:45:57.631646 4385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:45:57.632892 4385 hybrid_clock.cc:648] HybridClock initialized: now 1786423557632865 us; error 44 us; skew 500 ppm
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:45:57.636157 4385 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:45:57.637303 4385 webserver.cc:460] Webserver started at http://127.22.144.190:42245/ using document root <none> and password file <none>
I20260811 04:45:57.638042 4385 fs_manager.cc:359] Metadata directory not provided
I20260811 04:45:57.638155 4385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:45:57.638422 4385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:45:57.640751 4385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "0706fb975eee4a019659639d34b904d4"
format_stamp: "Formatted at 2026-08-11 04:45:57 on dist-test-slave-ss4n"
server_key: "74ec81d97e68b98b68dfad861f2e58b0"
server_key_iv: "5cb013061fc5c5899db409bf7d100ba4"
server_key_version: "encryptionkey@0"
I20260811 04:45:57.641415 4385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "0706fb975eee4a019659639d34b904d4"
format_stamp: "Formatted at 2026-08-11 04:45:57 on dist-test-slave-ss4n"
server_key: "74ec81d97e68b98b68dfad861f2e58b0"
server_key_iv: "5cb013061fc5c5899db409bf7d100ba4"
server_key_version: "encryptionkey@0"
I20260811 04:45:57.645756 4385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:45:57.648617 4400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:45:57.650101 4385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:45:57.650280 4385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "0706fb975eee4a019659639d34b904d4"
format_stamp: "Formatted at 2026-08-11 04:45:57 on dist-test-slave-ss4n"
server_key: "74ec81d97e68b98b68dfad861f2e58b0"
server_key_iv: "5cb013061fc5c5899db409bf7d100ba4"
server_key_version: "encryptionkey@0"
I20260811 04:45:57.650416 4385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:45:57.669103 4385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:45:57.672937 4385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:45:57.673272 4385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:45:57.681042 4385 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:36171
I20260811 04:45:57.681371 4470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:36171 every 8 connection(s)
I20260811 04:45:57.682194 4385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:45:57.682899 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 4385
I20260811 04:45:57.683069 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:45:57.683396 23106 external_mini_cluster.cc:1428] Setting key 5ec6abf3544293a142f587ac3504729a
I20260811 04:45:57.685400 4471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:45:57.689405 4471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.144.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.189@KRBTEST.COM" created.
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:45:57.708032 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.692960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.22.144.190:36171)
0811 04:45:57.693435 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:57.693452 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:57.693535 (+ 83us) server_negotiation.cc:407] Connection header received
0811 04:45:57.694244 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:57.694291 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:57.694527 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:57.694750 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:57.695661 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.696570 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.697801 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.698501 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.698709 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.702122 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:57.702129 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:57.702141 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:57.702187 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:57.704345 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.705065 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.705070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.705072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.705201 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.705675 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.705677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.705679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.706114 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:57.706230 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:57.706808 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 04:45:57.707060 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":337,"thread_start_us":118,"threads_started":1}
I20260811 04:45:57.708037 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.691838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36171 (local address 127.0.0.1:48552)
0811 04:45:57.692560 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.692624 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:57.693733 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:57.694800 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:57.694805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:57.695190 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:57.695455 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.695468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.696720 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.696723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.697671 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.697682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.698707 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.698711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.698837 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.699650 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:57.699681 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:57.701907 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:57.704497 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.704519 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.704535 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.704912 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.705317 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.705322 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.705326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.705538 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.706255 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:57.706266 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:57.706475 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 04:45:57.706684 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 04:45:57.707175 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":97,"thread_start_us":261,"threads_started":1}
I20260811 04:45:57.710819 4471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:45:57.711773 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.711454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45214)
0811 04:45:57.711605 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.711702 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 04:45:57.712040 4404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
I20260811 04:45:57.713105 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.712890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45230)
0811 04:45:57.712994 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.713049 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 04:45:57.713253 4471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:36819: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab.
Entry for principal kudu/127.22.144.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab.
I20260811 04:45:57.717067 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:36819
--webserver_interface=127.22.144.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab
--principal=kudu/127.22.144.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:36171,127.22.144.189:36819,127.22.144.188:41375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:45:57.762625 4471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } attempt: 1
I20260811 04:45:57.763779 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.763403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45246)
0811 04:45:57.763614 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.763686 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 04:45:57.764614 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.764382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45258)
0811 04:45:57.764499 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.764548 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 04:45:57.764807 4471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:36819: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
I20260811 04:45:57.833060 4471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } attempt: 2
I20260811 04:45:57.834038 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.833717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45262)
0811 04:45:57.833888 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.833956 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 04:45:57.834846 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.834607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45266)
0811 04:45:57.834742 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.834791 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 04:45:57.835022 4471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:36819: Network error: Client connection negotiation failed: client connection to 127.22.144.189:36819: connect: Connection refused (error 111)
W20260811 04:45:57.840448 4480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:45:57.840719 4480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:45:57.840761 4480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:45:57.844955 4480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:45:57.845016 4480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:45:57.845036 4480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:45:57.845052 4480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:45:57.845067 4480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:45:57.850548 4480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:36171,127.22.144.189:36819,127.22.144.188:41375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab
--principal=kudu/127.22.144.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:36819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.144.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:45:57.851651 4480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:45:57.852727 4480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:45:57.861899 4490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:45:57.862107 4487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:45:57.862202 4480 server_base.cc:1034] running on GCE node
W20260811 04:45:57.861917 4488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:45:57.862870 4480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:45:57.863936 4480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:45:57.865195 4480 hybrid_clock.cc:648] HybridClock initialized: now 1786423557865160 us; error 49 us; skew 500 ppm
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:45:57.868335 4480 init.cc:377] Logged in from keytab as kudu/127.22.144.189@KRBTEST.COM (short username kudu)
I20260811 04:45:57.869426 4480 webserver.cc:460] Webserver started at http://127.22.144.189:34701/ using document root <none> and password file <none>
I20260811 04:45:57.870116 4480 fs_manager.cc:359] Metadata directory not provided
I20260811 04:45:57.870224 4480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:45:57.870491 4480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:45:57.872766 4480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data/instance:
uuid: "c53147743ac247878445fe5be36e338a"
format_stamp: "Formatted at 2026-08-11 04:45:57 on dist-test-slave-ss4n"
server_key: "df5620293e42969d33af9e4b7e9df70d"
server_key_iv: "a69ae97d30332b2788f6d7843ca96cf0"
server_key_version: "encryptionkey@0"
I20260811 04:45:57.873413 4480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal/instance:
uuid: "c53147743ac247878445fe5be36e338a"
format_stamp: "Formatted at 2026-08-11 04:45:57 on dist-test-slave-ss4n"
server_key: "df5620293e42969d33af9e4b7e9df70d"
server_key_iv: "a69ae97d30332b2788f6d7843ca96cf0"
server_key_version: "encryptionkey@0"
I20260811 04:45:57.877836 4480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:45:57.880668 4497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:45:57.882365 4480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:45:57.882519 4480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal
uuid: "c53147743ac247878445fe5be36e338a"
format_stamp: "Formatted at 2026-08-11 04:45:57 on dist-test-slave-ss4n"
server_key: "df5620293e42969d33af9e4b7e9df70d"
server_key_iv: "a69ae97d30332b2788f6d7843ca96cf0"
server_key_version: "encryptionkey@0"
I20260811 04:45:57.882701 4480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:45:57.901762 4480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:45:57.905452 4480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:45:57.905769 4480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:45:57.913383 4480 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.189:36819
I20260811 04:45:57.913753 4558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.189:36819 every 8 connection(s)
I20260811 04:45:57.914834 4480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
I20260811 04:45:57.916011 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 4480
I20260811 04:45:57.916157 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-1/wal/instance
I20260811 04:45:57.916499 23106 external_mini_cluster.cc:1428] Setting key f57c0a031468bcb71985b46154b7dd27
I20260811 04:45:57.919250 4560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:45:57.924741 4560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:45:57.926213 4471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } attempt: 3
WARNING: no policy specified for kudu/127.22.144.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.188@KRBTEST.COM" created.
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:45:57.940712 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.926760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45282)
0811 04:45:57.927026 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.927041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:57.927140 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:57.929384 (+ 2244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:57.929388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:57.929688 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:57.929825 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.929833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.931498 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.931509 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.932155 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.932164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.933391 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.933398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.933515 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.934169 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:57.934182 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:57.935792 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:57.938546 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.938550 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.938552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.938817 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.939219 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.939222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.939225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.939324 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.940079 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:57.940083 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:57.940208 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 04:45:57.940333 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 04:45:57.940500 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260811 04:45:57.942155 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.927350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36171 (local address 127.0.0.1:48556)
0811 04:45:57.927982 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.928040 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:57.928919 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:57.929899 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:57.929906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:57.930472 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:57.930746 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.930759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.931728 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.931734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.932869 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.932879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.933658 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.933665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.933791 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.934331 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:57.934353 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:57.936742 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:57.939282 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.939293 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.939303 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.939646 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.940009 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.940014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.940016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.940113 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.940611 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:57.940618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:57.940761 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 04:45:57.941569 (+ 808us) client_negotiation.cc:240] Negotiation successful
0811 04:45:57.941863 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":40,"spinlock_wait_cycles":53504,"thread_start_us":186,"threads_started":1}
I20260811 04:45:57.942158 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.927350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.22.144.189:36819)
0811 04:45:57.928041 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:57.928056 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:57.928138 (+ 82us) server_negotiation.cc:407] Connection header received
0811 04:45:57.928919 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:57.928946 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:57.929220 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:57.929411 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:57.929955 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.931364 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.932296 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.933239 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.933436 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.935977 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:57.935987 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:57.936006 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:57.936040 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:57.938411 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.938948 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.938955 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.938965 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.939102 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.939430 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.939436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.939437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.939914 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:57.940105 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:57.940536 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 04:45:57.940894 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":386,"thread_start_us":159,"threads_started":1}
I20260811 04:45:57.942338 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.926912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.22.144.190:36171)
0811 04:45:57.927864 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:57.927868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:57.928170 (+ 302us) server_negotiation.cc:407] Connection header received
0811 04:45:57.929190 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:57.929193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:57.929245 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:57.929327 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:57.930911 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.931591 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.932999 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.933297 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.933427 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.936898 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:57.936906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:57.936911 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:57.936947 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:57.939140 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.939780 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.939784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.939786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.939878 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.940215 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.940219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.940221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.940448 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:57.940548 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:57.941997 (+ 1449us) server_negotiation.cc:299] Negotiation successful
0811 04:45:57.942150 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869}
I20260811 04:45:57.943850 4560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:45:57.944620 4471 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:45:57.945585 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.945182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48494)
0811 04:45:57.945332 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.945399 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 04:45:57.946401 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.946100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48500)
0811 04:45:57.946272 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.946332 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 04:45:57.946616 4471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:41375: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Aug 11 04:45:57 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab.
Entry for principal kudu/127.22.144.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab.
I20260811 04:45:57.954263 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.944516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45292)
0811 04:45:57.945100 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.945114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:57.945202 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:57.945520 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:57.945523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:57.945791 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:57.945900 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.945906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.947050 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.947056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.947694 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:57.947704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.948200 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.948203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:57.948299 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.948921 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:57.948935 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:57.950411 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:57.952487 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.952491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.952494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.952731 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.953041 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:57.953045 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:57.953047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:57.953143 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:57.953628 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:57.953631 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:57.953738 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 04:45:57.953861 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 04:45:57.954052 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501}
I20260811 04:45:57.954263 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.944742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.22.144.189:36819)
0811 04:45:57.944854 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:57.944859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:57.945187 (+ 328us) server_negotiation.cc:407] Connection header received
0811 04:45:57.945323 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:57.945326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:57.945379 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:57.945438 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:57.946027 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.946918 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.947814 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:57.948073 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:57.948222 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:57.950559 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:57.950563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:57.950566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:57.950592 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:57.952310 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.952838 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.952844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.952846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.952935 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:57.953227 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:57.953233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:57.953235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:57.953468 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:57.953642 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:57.953925 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 04:45:57.954137 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":38}
I20260811 04:45:57.954523 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:41375
--webserver_interface=127.22.144.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab
--principal=kudu/127.22.144.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:36171,127.22.144.189:36819,127.22.144.188:41375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:45:57.955466 4560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:45:57.956317 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.956009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48504)
0811 04:45:57.956166 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.956254 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 04:45:57.957010 4501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
I20260811 04:45:57.958170 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.957884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48518)
0811 04:45:57.958022 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.958087 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 04:45:57.958355 4560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:41375: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
I20260811 04:45:57.977784 4471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } attempt: 1
I20260811 04:45:57.978780 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.978426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48534)
0811 04:45:57.978617 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.978692 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 04:45:57.979575 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:57.979340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48538)
0811 04:45:57.979469 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:57.979519 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 04:45:57.979737 4471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:41375: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
I20260811 04:45:58.007685 4560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } attempt: 1
I20260811 04:45:58.008677 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.008292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48548)
0811 04:45:58.008519 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.008590 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 04:45:58.009626 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.009304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48554)
0811 04:45:58.009422 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.009552 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 04:45:58.009817 4560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:41375: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
I20260811 04:45:58.054920 4471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } attempt: 2
I20260811 04:45:58.055964 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.055602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48568)
0811 04:45:58.055788 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.055879 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 04:45:58.056877 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.056644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48572)
0811 04:45:58.056771 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.056820 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 04:45:58.057062 4471 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:41375: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
W20260811 04:45:58.075416 4566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:45:58.075692 4566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:45:58.075733 4566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 04:45:58.078035 4560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } attempt: 2
I20260811 04:45:58.079034 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.078707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48580)
0811 04:45:58.078890 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.078956 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 04:45:58.079841 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.079642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48594)
0811 04:45:58.079741 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.079788 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 04:45:58.080052 4560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:41375: Network error: Client connection negotiation failed: client connection to 127.22.144.188:41375: connect: Connection refused (error 111)
W20260811 04:45:58.080467 4566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:45:58.080541 4566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:45:58.080607 4566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:45:58.080660 4566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:45:58.080696 4566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:45:58.086416 4566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:36171,127.22.144.189:36819,127.22.144.188:41375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab
--principal=kudu/127.22.144.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:41375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.144.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:45:58.087621 4566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:45:58.088665 4566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:45:58.097579 4576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:45:58.097589 4573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:45:58.097676 4574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:45:58.097783 4566 server_base.cc:1034] running on GCE node
I20260811 04:45:58.098377 4566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:45:58.099543 4566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:45:58.100736 4566 hybrid_clock.cc:648] HybridClock initialized: now 1786423558100720 us; error 32 us; skew 500 ppm
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423558, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:45:58.103806 4566 init.cc:377] Logged in from keytab as kudu/127.22.144.188@KRBTEST.COM (short username kudu)
I20260811 04:45:58.104699 4566 webserver.cc:460] Webserver started at http://127.22.144.188:41835/ using document root <none> and password file <none>
I20260811 04:45:58.105232 4566 fs_manager.cc:359] Metadata directory not provided
I20260811 04:45:58.105291 4566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:45:58.105472 4566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:45:58.107638 4566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data/instance:
uuid: "23c99da7de474588a273d9131a7a25ae"
format_stamp: "Formatted at 2026-08-11 04:45:58 on dist-test-slave-ss4n"
server_key: "4792357a96956b0b641dcd0c1318f979"
server_key_iv: "3230dadae68818b5af53ba7831122462"
server_key_version: "encryptionkey@0"
I20260811 04:45:58.108222 4566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal/instance:
uuid: "23c99da7de474588a273d9131a7a25ae"
format_stamp: "Formatted at 2026-08-11 04:45:58 on dist-test-slave-ss4n"
server_key: "4792357a96956b0b641dcd0c1318f979"
server_key_iv: "3230dadae68818b5af53ba7831122462"
server_key_version: "encryptionkey@0"
I20260811 04:45:58.112305 4566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 04:45:58.114948 4585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:45:58.116276 4566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:45:58.116427 4566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal
uuid: "23c99da7de474588a273d9131a7a25ae"
format_stamp: "Formatted at 2026-08-11 04:45:58 on dist-test-slave-ss4n"
server_key: "4792357a96956b0b641dcd0c1318f979"
server_key_iv: "3230dadae68818b5af53ba7831122462"
server_key_version: "encryptionkey@0"
I20260811 04:45:58.116544 4566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:45:58.138772 4566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:45:58.142376 4566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:45:58.142611 4566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:45:58.149869 4566 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.188:41375
I20260811 04:45:58.150185 4645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.188:41375 every 8 connection(s)
I20260811 04:45:58.150972 4566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
I20260811 04:45:58.154131 4646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:45:58.154264 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 4566
I20260811 04:45:58.154387 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786423274535345-23106-0/minicluster-data/master-2/wal/instance
I20260811 04:45:58.154706 23106 external_mini_cluster.cc:1428] Setting key 6db81f50bcbf41214e37e7263932d353
I20260811 04:45:58.156641 4471 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } attempt: 3
I20260811 04:45:58.159799 4646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423558, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:45:58.170497 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.157158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48616)
0811 04:45:58.157526 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.157545 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:58.157643 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:58.158617 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:58.158621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:58.158850 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:58.158962 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.158969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.160350 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.160353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.161174 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.161186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.162361 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.162365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.162460 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.163334 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:58.163349 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:58.165510 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:58.168395 (+ 2885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.168398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.168400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.168670 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.169019 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.169024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.169026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.169140 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.169931 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:58.169935 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:58.170038 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 04:45:58.170147 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 04:45:58.170308 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260811 04:45:58.171599 4560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } attempt: 3
I20260811 04:45:58.171787 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.156487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.22.144.188:41375)
0811 04:45:58.157045 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.157064 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.157122 (+ 58us) server_negotiation.cc:407] Connection header received
0811 04:45:58.157971 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.157987 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.158327 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.159558 (+ 1231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.159650 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.160390 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.161570 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.162223 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.162571 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.165745 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.165748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.165750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.165773 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.168130 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.168705 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.168712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.168719 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.168912 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.169260 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.169265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.169267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.169808 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.169997 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.170391 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.170830 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":396,"spinlock_wait_cycles":15872,"thread_start_us":136,"threads_started":1}
I20260811 04:45:58.171828 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.157173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.22.144.188:41375)
0811 04:45:58.157724 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.157732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.157744 (+ 12us) server_negotiation.cc:407] Connection header received
0811 04:45:58.157965 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.157984 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.158323 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.158699 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.159169 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.160142 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.161313 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.162223 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.162438 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.165713 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.165719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.165731 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.165760 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.168281 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.168786 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.168789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.168791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.168915 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.169262 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.169265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.169267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.169812 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.170043 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.170337 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.170920 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":451,"spinlock_wait_cycles":20352,"thread_start_us":141,"threads_started":1}
I20260811 04:45:58.175269 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.161675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.22.144.190:36171)
0811 04:45:58.161797 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.161801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.162726 (+ 925us) server_negotiation.cc:407] Connection header received
0811 04:45:58.162963 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.162967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.163066 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.163183 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.164355 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.164918 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.166238 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.166606 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.166729 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.171274 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.171278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.171281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.171305 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.173099 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.173692 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.173698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.173703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.173807 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.174154 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.174157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.174158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.174398 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.174494 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.174924 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.175079 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 04:45:58.176052 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.161644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36171 (local address 127.0.0.1:48566)
0811 04:45:58.162601 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.162654 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:58.162818 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:58.163435 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:58.163443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:58.164039 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:58.164211 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.164220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.165036 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.165040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.166062 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.166076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.166863 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.166866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.167012 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.167521 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:58.167536 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:58.171133 (+ 3597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:58.173221 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.173228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.173233 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.173558 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.173924 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.173927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.173929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.174028 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.174554 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:58.174560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:58.174694 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 04:45:58.175688 (+ 994us) client_negotiation.cc:240] Negotiation successful
0811 04:45:58.175890 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":97,"threads_started":1}
I20260811 04:45:58.176596 4471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: Bootstrap starting.
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:45:58.179751 4471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: Neither blocks nor log segments found. Creating new log.
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:45:58.180758 4471 log.cc:824] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: Log is configured to *not* fsync() on all Append() calls
I20260811 04:45:58.182567 4646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:45:58.182961 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.172070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48620)
0811 04:45:58.172333 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.172348 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:58.172429 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:58.173258 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:58.173261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:58.173444 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:58.173597 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.173605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.175487 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.175493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.176290 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.176303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.176912 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.176916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.177002 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.177541 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:58.177556 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:58.179002 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:58.181049 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.181052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.181054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.181370 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.181750 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.181754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.181756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.181849 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.182353 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:58.182356 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:58.182462 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 04:45:58.182547 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 04:45:58.182787 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260811 04:45:58.183634 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.172898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.22.144.188:41375)
0811 04:45:58.173012 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.173016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.173028 (+ 12us) server_negotiation.cc:407] Connection header received
0811 04:45:58.173060 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.173063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.173103 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.173720 (+ 617us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.173806 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.175345 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.176466 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.176773 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.177233 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.179165 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.179169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.179171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.179195 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.180921 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.181538 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.181541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.181543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.181625 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.181993 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.181996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.181998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.182231 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.183259 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.183339 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.183456 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:45:58.184340 4471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: No bootstrap required, opened a new log
I20260811 04:45:58.185672 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.174511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.22.144.190:36171)
0811 04:45:58.174829 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.174845 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.174859 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:45:58.174892 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.174895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.174935 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.175889 (+ 954us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.175985 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.176672 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.177715 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.177987 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.178095 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.180686 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.180691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.180693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.180718 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.182533 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.183397 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.183402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.183406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.183504 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.184241 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.184244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.184245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.184532 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.184735 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.185355 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.185497 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":83,"threads_started":1}
I20260811 04:45:58.187759 4560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: Bootstrap starting.
I20260811 04:45:58.188635 4471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.188889 4471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:45:58.188961 4471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0706fb975eee4a019659639d34b904d4, State: Initialized, Role: FOLLOWER
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423558, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
I20260811 04:45:58.189474 4471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.190661 4560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: Neither blocks nor log segments found. Creating new log.
I20260811 04:45:58.191051 4471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:45:58.191633 4560 log.cc:824] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: Log is configured to *not* fsync() on all Append() calls
I20260811 04:45:58.193547 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.183105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:36819 (local address 127.0.0.1:45298)
0811 04:45:58.183262 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.183275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:58.183350 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:58.183682 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:58.183685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:58.183849 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:58.183941 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.183946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.184672 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.184677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.185625 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.185637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.186596 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.186603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.186826 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.188064 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:58.188078 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:58.189613 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:58.191758 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.191761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.191763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.192021 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.192365 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.192367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.192370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.192466 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.193011 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:58.193014 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:58.193112 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 04:45:58.193217 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 04:45:58.193355 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 04:45:58.194026 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.183183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.22.144.189:36819)
0811 04:45:58.183292 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.183297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.183310 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:45:58.183448 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.183451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.183496 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.183572 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.184065 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.184552 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.185757 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.186053 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.186138 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.189777 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.189781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.189784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.189809 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.191618 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.192165 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.192170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.192173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.192266 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.192577 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.192581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.192583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.192852 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.192960 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.193776 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.193894 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 04:45:58.194352 4560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: No bootstrap required, opened a new log
I20260811 04:45:58.194674 4646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
Aug 11 04:45:58 dist-test-slave-ss4n krb5kdc[4367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423558, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:45:58.196472 4385 master_runner.cc:186] Error getting master registration for 127.22.144.189:36819: OK, instance_id { permanent_uuid: "c53147743ac247878445fe5be36e338a" instance_seqno: 1786423557911252 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 04:45:58.200800 4385 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 04:45:58.199856 4560 raft_consensus.cc:348] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.200975 4560 raft_consensus.cc:374] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:45:58.201045 4560 raft_consensus.cc:729] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53147743ac247878445fe5be36e338a, State: Initialized, Role: FOLLOWER
I20260811 04:45:58.201663 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.187312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.22.144.189:36819)
0811 04:45:58.189846 (+ 2534us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.189863 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.189879 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:45:58.189919 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.189922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.189969 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.191265 (+ 1296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.191369 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.192300 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.193467 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.194288 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.194853 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.197376 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.197381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.197383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.197408 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.199275 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.199830 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.199834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.199836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.199977 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.200293 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.200294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.200296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.200467 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.200671 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.201319 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.201446 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2345,"thread_start_us":151,"threads_started":1}
I20260811 04:45:58.201663 4560 consensus_queue.cc:260] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.203253 4560 sys_catalog.cc:564] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:45:58.204036 4670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.204221 4670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:45:58.205247 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:45:58.205633 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.195208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:41375 (local address 127.0.0.1:48630)
0811 04:45:58.195507 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 04:45:58.195523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:45:58.195593 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:45:58.196002 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:45:58.196004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:45:58.196223 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:45:58.196329 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.196337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.197397 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.197401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.198266 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:45:58.198280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.198920 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.198923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:45:58.198996 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.199460 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:45:58.199476 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:45:58.201346 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:45:58.203730 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.203733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.203735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.203967 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.204540 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:45:58.204543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:45:58.204545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:45:58.204635 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:45:58.205127 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:45:58.205129 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:45:58.205214 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 04:45:58.205315 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 04:45:58.205455 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
W20260811 04:45:58.205622 4669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:45:58.205690 4669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:45:58.205902 4655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.206030 4655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:45:58.206228 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.195411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.22.144.188:41375)
0811 04:45:58.195752 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.195757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.195769 (+ 12us) server_negotiation.cc:407] Connection header received
0811 04:45:58.195797 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.195800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.195839 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.195909 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.196445 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.197265 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.198404 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.198727 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.198815 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.201505 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.201510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.201513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.201537 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.203613 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.204075 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.204078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.204083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.204426 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.204732 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.204735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.204737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.204945 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.205054 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.205950 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.206081 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 04:45:58.209827 4646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: Bootstrap starting.
I20260811 04:45:58.212774 4646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: Neither blocks nor log segments found. Creating new log.
I20260811 04:45:58.213881 4646 log.cc:824] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: Log is configured to *not* fsync() on all Append() calls
I20260811 04:45:58.211360 4480 master_runner.cc:186] Error getting master registration for 127.22.144.188:41375: OK, instance_id { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" instance_seqno: 1786423558147779 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 04:45:58.214955 4480 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 04:45:58.218667 4682 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:45:58.218747 4682 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:45:58.220116 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.208618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48582 (local address 127.22.144.190:36171)
0811 04:45:58.208744 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.208750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.210084 (+ 1334us) server_negotiation.cc:407] Connection header received
0811 04:45:58.210291 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.210295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.210344 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.210398 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.211466 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.212268 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.213592 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.213949 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.214032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.216166 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.216170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.216173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.216203 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.218056 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.218586 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.218590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.218592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.218685 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.219083 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.219086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.219088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.219355 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.219514 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.219806 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.219946 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 04:45:58.223731 4646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: No bootstrap required, opened a new log
I20260811 04:45:58.228567 4646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.228879 4646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:45:58.228940 4646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c99da7de474588a273d9131a7a25ae, State: Initialized, Role: FOLLOWER
I20260811 04:45:58.229717 4646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.230244 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.208995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.22.144.188:41375)
0811 04:45:58.209097 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.209101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.221053 (+ 11952us) server_negotiation.cc:407] Connection header received
0811 04:45:58.221108 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.221110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.221165 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.221250 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.222238 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.223030 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.224190 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.224535 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.224623 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.226429 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.226433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.226436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.226466 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.228283 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.228791 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.228795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.228796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.228882 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.229196 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.229199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.229201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.229426 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.229580 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.229958 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.230082 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 04:45:58.231624 4646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:45:58.231338 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:45:58.215109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.22.144.189:36819)
0811 04:45:58.215241 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 04:45:58.215247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:45:58.221778 (+ 6531us) server_negotiation.cc:407] Connection header received
0811 04:45:58.221973 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:45:58.221978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:45:58.222035 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:45:58.222096 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:45:58.223080 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.223812 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.224877 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:45:58.225216 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:45:58.225294 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:45:58.227653 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:45:58.227659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:45:58.227662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:45:58.227692 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:45:58.229404 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.229936 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.229942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.229943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.230036 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:45:58.230373 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:45:58.230378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:45:58.230381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:45:58.230613 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:45:58.230734 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 04:45:58.231067 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 04:45:58.231197 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:45:58.234815 4695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.234934 4695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: This master's current role is: FOLLOWER
W20260811 04:45:58.237612 4709 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:45:58.237699 4709 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:45:58.240903 4566 master_runner.cc:263] No leader master found from master 23c99da7de474588a273d9131a7a25ae
I20260811 04:45:58.240984 4566 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 04:45:58.394025 4592 tablet.cc:2369] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:45:58.407683 4504 tablet.cc:2369] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:45:58.425745 4407 tablet.cc:2369] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:45:58.490919 4655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:45:58.491171 4655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.492233 4655 leader_election.cc:288] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c53147743ac247878445fe5be36e338a (127.22.144.189:36819), 23c99da7de474588a273d9131a7a25ae (127.22.144.188:41375)
I20260811 04:45:58.493023 4615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0706fb975eee4a019659639d34b904d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c99da7de474588a273d9131a7a25ae" is_pre_election: true
I20260811 04:45:58.493028 4527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0706fb975eee4a019659639d34b904d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53147743ac247878445fe5be36e338a" is_pre_election: true
I20260811 04:45:58.493387 4527 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0706fb975eee4a019659639d34b904d4 in term 0.
I20260811 04:45:58.493391 4615 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0706fb975eee4a019659639d34b904d4 in term 0.
I20260811 04:45:58.493955 4403 leader_election.cc:302] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0706fb975eee4a019659639d34b904d4, 23c99da7de474588a273d9131a7a25ae, c53147743ac247878445fe5be36e338a; no voters:
I20260811 04:45:58.494289 4655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:45:58.494380 4655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:45:58.494426 4655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:45:58.495529 4655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.495895 4655 leader_election.cc:288] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [CANDIDATE]: Term 1 election: Requested vote from peers c53147743ac247878445fe5be36e338a (127.22.144.189:36819), 23c99da7de474588a273d9131a7a25ae (127.22.144.188:41375)
I20260811 04:45:58.496277 4615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0706fb975eee4a019659639d34b904d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c99da7de474588a273d9131a7a25ae"
I20260811 04:45:58.496297 4527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0706fb975eee4a019659639d34b904d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c53147743ac247878445fe5be36e338a"
I20260811 04:45:58.496438 4615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:45:58.496482 4527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:45:58.497601 4615 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0706fb975eee4a019659639d34b904d4 in term 1.
I20260811 04:45:58.497833 4527 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0706fb975eee4a019659639d34b904d4 in term 1.
I20260811 04:45:58.497906 4403 leader_election.cc:302] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0706fb975eee4a019659639d34b904d4, 23c99da7de474588a273d9131a7a25ae; no voters:
I20260811 04:45:58.498070 4655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:45:58.498287 4655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [term 1 LEADER]: Becoming Leader. State: Replica: 0706fb975eee4a019659639d34b904d4, State: Running, Role: LEADER
I20260811 04:45:58.498638 4655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } }
I20260811 04:45:58.500661 4711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0706fb975eee4a019659639d34b904d4. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.500830 4711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: This master's current role is: LEADER
I20260811 04:45:58.501271 4719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:45:58.502481 4719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:45:58.507241 4615 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [term 1 FOLLOWER]: Refusing update from remote peer 0706fb975eee4a019659639d34b904d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:45:58.507308 4527 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [term 1 FOLLOWER]: Refusing update from remote peer 0706fb975eee4a019659639d34b904d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:45:58.507943 4655 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:45:58.508142 4711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:45:58.512118 4695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0706fb975eee4a019659639d34b904d4. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.512130 4670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0706fb975eee4a019659639d34b904d4. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.512261 4670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:45:58.512261 4695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:45:58.515528 4725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.515779 4725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: This master's current role is: LEADER
I20260811 04:45:58.515813 4670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.515815 4695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.515931 4695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:45:58.515930 4670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:45:58.516112 4655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0706fb975eee4a019659639d34b904d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0706fb975eee4a019659639d34b904d4" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36171 } } peers { permanent_uuid: "c53147743ac247878445fe5be36e338a" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 36819 } } peers { permanent_uuid: "23c99da7de474588a273d9131a7a25ae" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 41375 } } }
I20260811 04:45:58.516152 4721 mvcc.cc:205] Tried to move back new op lower bound from 7317190895639207936 to 7317190895612919808. Current Snapshot: MvccSnapshot[applied={T|T < 7317190895639207936}]
I20260811 04:45:58.516220 4655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4 [sys.catalog]: This master's current role is: LEADER
I20260811 04:45:58.517962 4719 catalog_manager.cc:1347] Generated new cluster ID: da0b427facdf48c18e492a2fdb1ffafe
I20260811 04:45:58.518046 4719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:45:58.534171 4719 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:45:58.535567 4719 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:45:58.556820 4719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0706fb975eee4a019659639d34b904d4: Generated new TSK 0
I20260811 04:45:58.557861 4719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:45:59.220360 4682 catalog_manager.cc:1259] Loaded cluster ID: da0b427facdf48c18e492a2fdb1ffafe
I20260811 04:45:59.220469 4682 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: loading cluster ID for follower catalog manager: success
I20260811 04:45:59.222332 4682 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: acquiring CA information for follower catalog manager: success
I20260811 04:45:59.223531 4682 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c53147743ac247878445fe5be36e338a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:45:59.239461 4709 catalog_manager.cc:1259] Loaded cluster ID: da0b427facdf48c18e492a2fdb1ffafe
I20260811 04:45:59.239567 4709 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: loading cluster ID for follower catalog manager: success
I20260811 04:45:59.242470 4709 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: acquiring CA information for follower catalog manager: success
I20260811 04:45:59.244482 4709 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23c99da7de474588a273d9131a7a25ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:45:59.635033 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 4385
I20260811 04:45:59.645025 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 4480
I20260811 04:45:59.652510 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 4566
2026-08-11T04:45:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3503 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:45:59 dist-test-slave-ss4n krb5kdc[4736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:45:59 dist-test-slave-ss4n krb5kdc[4736](info): set up 2 sockets
Aug 11 04:45:59 dist-test-slave-ss4n krb5kdc[4736](info): commencing operation
krb5kdc: starting...
W20260811 04:46:00.765848 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:46:00 dist-test-slave-ss4n krb5kdc[4736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:46:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:46:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:46:00.903270 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42967
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:01.024799 4750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:01.025132 4750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:01.025215 4750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:01.029346 4750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:46:01.029453 4750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:01.029492 4750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:01.029556 4750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:46:01.029619 4750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:46:01.034961 4750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:01.036124 4750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:01.037326 4750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:01.046727 4758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:01.046732 4756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:01.046730 4760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:01.046972 4750 server_base.cc:1034] running on GCE node
I20260811 04:46:01.047672 4750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:01.048852 4750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:01.050066 4750 hybrid_clock.cc:648] HybridClock initialized: now 1786423561050044 us; error 28 us; skew 500 ppm
Aug 11 04:46:01 dist-test-slave-ss4n krb5kdc[4736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423561, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:01.053062 4750 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:46:01.054240 4750 webserver.cc:460] Webserver started at http://127.22.144.190:45077/ using document root <none> and password file <none>
I20260811 04:46:01.054852 4750 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:01.054947 4750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:01.055224 4750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:01.057438 4750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "c1a48fdb2e3b47e5b4bf07d56e619186"
format_stamp: "Formatted at 2026-08-11 04:46:01 on dist-test-slave-ss4n"
server_key: "830a23858fa35c978036c648c1f3b606"
server_key_iv: "ee7277bec767255331dd4fd1428b60ba"
server_key_version: "encryptionkey@0"
I20260811 04:46:01.058131 4750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "c1a48fdb2e3b47e5b4bf07d56e619186"
format_stamp: "Formatted at 2026-08-11 04:46:01 on dist-test-slave-ss4n"
server_key: "830a23858fa35c978036c648c1f3b606"
server_key_iv: "ee7277bec767255331dd4fd1428b60ba"
server_key_version: "encryptionkey@0"
I20260811 04:46:01.062212 4750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:46:01.065044 4768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:01.066659 4750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:01.066846 4750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "c1a48fdb2e3b47e5b4bf07d56e619186"
format_stamp: "Formatted at 2026-08-11 04:46:01 on dist-test-slave-ss4n"
server_key: "830a23858fa35c978036c648c1f3b606"
server_key_iv: "ee7277bec767255331dd4fd1428b60ba"
server_key_version: "encryptionkey@0"
I20260811 04:46:01.067001 4750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:01.077425 4750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:01.080768 4750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:01.081018 4750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:01.089349 4750 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:42967
I20260811 04:46:01.089378 4829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:42967 every 8 connection(s)
I20260811 04:46:01.090440 4750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:46:01.091336 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 4750
I20260811 04:46:01.091446 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:46:01.091789 23106 external_mini_cluster.cc:1428] Setting key a92009afa58976bdaa1cec62ebd99c2c
I20260811 04:46:01.093932 4830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:01.100142 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186: Bootstrap starting.
Aug 11 04:46:01 dist-test-slave-ss4n krb5kdc[4736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:01.103238 4830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:01.104066 4830 log.cc:824] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:01.106837 4830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186: No bootstrap required, opened a new log
I20260811 04:46:01.108002 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:01.093285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.22.144.190:42967)
0811 04:46:01.093784 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:01.093801 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:01.093840 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:46:01.094594 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:01.094615 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:01.094972 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:01.095340 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:01.096239 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:01.097359 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:01.098342 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:01.098869 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:01.099072 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:01.102239 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:01.102244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:01.102260 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:01.102293 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:01.104262 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:01.104843 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:01.104849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:01.104854 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:01.105025 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:01.105365 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:01.105368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:01.105370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:01.106112 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:01.106321 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:01.106624 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 04:46:01.106956 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":183,"threads_started":1}
I20260811 04:46:01.111442 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:46:01.111732 4830 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42967 } }
I20260811 04:46:01.112053 4830 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:01.112125 4830 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1a48fdb2e3b47e5b4bf07d56e619186, State: Initialized, Role: FOLLOWER
I20260811 04:46:01.112610 4830 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42967 } }
I20260811 04:46:01.112790 4830 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:46:01.112876 4830 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:46:01.113029 4830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:01.114153 4830 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42967 } }
I20260811 04:46:01.114527 4830 leader_election.cc:302] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1a48fdb2e3b47e5b4bf07d56e619186; no voters:
I20260811 04:46:01.114868 4830 leader_election.cc:288] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:46:01.115008 4836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:01.115281 4836 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [term 1 LEADER]: Becoming Leader. State: Replica: c1a48fdb2e3b47e5b4bf07d56e619186, State: Running, Role: LEADER
I20260811 04:46:01.115644 4836 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42967 } }
I20260811 04:46:01.116196 4830 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:46:01.117419 4837 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42967 } } }
I20260811 04:46:01.117614 4837 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:01.117628 4838 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1a48fdb2e3b47e5b4bf07d56e619186. Latest consensus state: current_term: 1 leader_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1a48fdb2e3b47e5b4bf07d56e619186" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 42967 } } }
I20260811 04:46:01.117751 4838 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186 [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:01.118114 4846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:46:01.121727 4846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:46:01.127866 4846 catalog_manager.cc:1347] Generated new cluster ID: 8bcb01876e314641b277ef84ab61dae9
I20260811 04:46:01.127938 4846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:46:01.145288 4846 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:46:01.146312 4846 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:46:01.157936 4846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1a48fdb2e3b47e5b4bf07d56e619186: Generated new TSK 0
I20260811 04:46:01.158687 4846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:46:01.168794 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 4750
2026-08-11T04:46:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1514 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:46:01 dist-test-slave-ss4n krb5kdc[4860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:46:01 dist-test-slave-ss4n krb5kdc[4860](info): set up 2 sockets
Aug 11 04:46:01 dist-test-slave-ss4n krb5kdc[4860](info): commencing operation
krb5kdc: starting...
W20260811 04:46:02.324836 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:46:02 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:46:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:46:02.461366 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41843
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:02.577668 4877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:02.577992 4877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:02.578068 4877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:02.582157 4877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:46:02.582281 4877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:02.582319 4877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:02.582379 4877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:46:02.582417 4877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:46:02.587859 4877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:02.588985 4877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:02.590117 4877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:02.599200 4884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:02.599202 4883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:02.599357 4877 server_base.cc:1034] running on GCE node
W20260811 04:46:02.599200 4887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:02.600320 4877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:02.601637 4877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:02.602835 4877 hybrid_clock.cc:648] HybridClock initialized: now 1786423562602821 us; error 29 us; skew 500 ppm
Aug 11 04:46:02 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423562, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:02.605773 4877 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:46:02.606647 4877 webserver.cc:460] Webserver started at http://127.22.144.190:44837/ using document root <none> and password file <none>
I20260811 04:46:02.607193 4877 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:02.607259 4877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:02.607465 4877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:02.609738 4877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "733a1b1464144dd29e36fb84388d164b"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "c7e376fe71471d6995f2a04729be3c8b"
server_key_iv: "7e3c467c9d0532474631ddf73ae8000d"
server_key_version: "encryptionkey@0"
I20260811 04:46:02.610286 4877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "733a1b1464144dd29e36fb84388d164b"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "c7e376fe71471d6995f2a04729be3c8b"
server_key_iv: "7e3c467c9d0532474631ddf73ae8000d"
server_key_version: "encryptionkey@0"
I20260811 04:46:02.614189 4877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:46:02.616647 4896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:02.617967 4877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:46:02.618099 4877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "733a1b1464144dd29e36fb84388d164b"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "c7e376fe71471d6995f2a04729be3c8b"
server_key_iv: "7e3c467c9d0532474631ddf73ae8000d"
server_key_version: "encryptionkey@0"
I20260811 04:46:02.618247 4877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:02.649113 4877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:02.652848 4877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:02.653080 4877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:02.660754 4877 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:41843
I20260811 04:46:02.661070 4959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:41843 every 8 connection(s)
I20260811 04:46:02.661904 4877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:46:02.664822 4960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:02.669221 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 4877
I20260811 04:46:02.669338 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:46:02.669690 23106 external_mini_cluster.cc:1428] Setting key edc95cd45b6d3743bfd88a6d039416a1
I20260811 04:46:02.670993 4960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap starting.
I20260811 04:46:02.673942 4960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:02.674925 4960 log.cc:824] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:02.677186 4960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: No bootstrap required, opened a new log
Aug 11 04:46:02 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:02.681396 4960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:02.681799 4960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:02.681893 4960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733a1b1464144dd29e36fb84388d164b, State: Initialized, Role: FOLLOWER
I20260811 04:46:02.682451 4960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:02.682627 4960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:46:02.682715 4960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:46:02.682832 4960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:02.683904 4960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:02.684486 4960 leader_election.cc:302] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 733a1b1464144dd29e36fb84388d164b; no voters:
I20260811 04:46:02.684877 4960 leader_election.cc:288] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:46:02.685002 4965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:02.685235 4965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 LEADER]: Becoming Leader. State: Replica: 733a1b1464144dd29e36fb84388d164b, State: Running, Role: LEADER
I20260811 04:46:02.685678 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:02.671270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.22.144.190:41843)
0811 04:46:02.671725 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:02.671744 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:02.671787 (+ 43us) server_negotiation.cc:407] Connection header received
0811 04:46:02.672709 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:02.672734 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:02.673155 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:02.673578 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:02.674339 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:02.675708 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:02.676587 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:02.677086 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:02.677282 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:02.679985 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:02.679990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:02.680006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:02.680036 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:02.682189 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:02.682669 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:02.682677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:02.682687 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:02.682863 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:02.683174 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:02.683181 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:02.683184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:02.683618 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:02.683853 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:02.684158 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 04:46:02.684501 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
I20260811 04:46:02.685648 4965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:02.686287 4960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:46:02.688825 4966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "733a1b1464144dd29e36fb84388d164b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } } }
I20260811 04:46:02.689152 4967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 733a1b1464144dd29e36fb84388d164b. Latest consensus state: current_term: 1 leader_uuid: "733a1b1464144dd29e36fb84388d164b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } } }
I20260811 04:46:02.689246 4967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:02.689246 4966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: This master's current role is: LEADER
W20260811 04:46:02.691687 4981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:46:02.691828 4981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:46:02.691963 4985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:46:02.692792 4985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:46:02.698680 4985 catalog_manager.cc:1347] Generated new cluster ID: 60cb661fff0d4332879ef0767ff51757
I20260811 04:46:02.698763 4985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:46:02.726784 4985 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:46:02.727766 4985 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:46:02.745227 4985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Generated new TSK 0
I20260811 04:46:02.746068 4985 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:46:02.786345 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41843
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:02.902289 4991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:02.902570 4991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:02.902614 4991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:02.907563 4991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:02.907635 4991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:02.907696 4991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:46:02.913141 4991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:02.914386 4991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:02.915406 4991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:02.925908 5003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:02.925927 4997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:02.926008 4991 server_base.cc:1034] running on GCE node
W20260811 04:46:02.925935 4998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:02.926640 4991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:02.927281 4991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:02.928455 4991 hybrid_clock.cc:648] HybridClock initialized: now 1786423562928424 us; error 33 us; skew 500 ppm
Aug 11 04:46:02 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423562, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:02.931375 4991 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:46:02.932382 4991 webserver.cc:460] Webserver started at http://127.22.144.129:44033/ using document root <none> and password file <none>
I20260811 04:46:02.933007 4991 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:02.933113 4991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:02.933372 4991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:02.935909 4991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "5e09262d707737001b430383f4a327e4"
server_key_iv: "fc6847f6e5e2ccffeba2a69d415ea6da"
server_key_version: "encryptionkey@0"
I20260811 04:46:02.936550 4991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "5e09262d707737001b430383f4a327e4"
server_key_iv: "fc6847f6e5e2ccffeba2a69d415ea6da"
server_key_version: "encryptionkey@0"
I20260811 04:46:02.940644 4991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 04:46:02.943382 5011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:02.944787 4991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:02.944947 4991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "5e09262d707737001b430383f4a327e4"
server_key_iv: "fc6847f6e5e2ccffeba2a69d415ea6da"
server_key_version: "encryptionkey@0"
I20260811 04:46:02.945079 4991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:02.975805 4991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:02.979168 4991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:02.979393 4991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:02.980027 4991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:02.980926 4991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:02.981001 4991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:02.981081 4991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:02.981142 4991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:02.991027 4991 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:45537
I20260811 04:46:02.991050 5153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:45537 every 8 connection(s)
I20260811 04:46:02.992057 4991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:46:02.994709 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 4991
I20260811 04:46:02.994959 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:46:02.995283 23106 external_mini_cluster.cc:1428] Setting key 74230c075a5d1d2a316929a9de890dce
Aug 11 04:46:03 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423562, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:46:03.007751 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:02.994025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:45083 (local address 127.22.144.190:41843)
0811 04:46:02.994194 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:02.994199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:02.995081 (+ 882us) server_negotiation.cc:407] Connection header received
0811 04:46:02.996915 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:02.996918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:02.996970 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:02.997060 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:02.998385 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:02.999260 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.000436 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.000681 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.000790 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.003807 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:03.003810 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:03.003812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:03.003838 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:03.005450 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.006089 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.006097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.006099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.006180 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.006543 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.006549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.006550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.006802 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:03.006918 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.007444 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.007580 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 04:46:03.008667 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:02.994457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:45083)
0811 04:46:02.994928 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:02.994982 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:02.996664 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:02.997219 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:02.997226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:02.997742 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:02.998235 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:02.998248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:02.999397 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:02.999400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.000298 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.000308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.000846 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.000850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.001032 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.001746 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:03.001774 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:03.003621 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:03.005628 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:03.005635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:03.005651 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:03.005961 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:03.006281 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:03.006284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:03.006286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:03.006443 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:03.006919 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:03.006927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:03.007200 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.007404 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.007767 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:46:03.010103 5154 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41843
I20260811 04:46:03.010380 5154 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:03.010923 5154 heartbeater.cc:511] Master 127.22.144.190:41843 requested a full tablet report, sending...
I20260811 04:46:03.012805 4916 ts_manager.cc:194] Registered new tserver with Master: f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129:45537)
I20260811 04:46:03.014515 4916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:45083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:46:03.028877 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41843
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:03.143630 5162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:03.143953 5162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:03.144034 5162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:03.148092 5162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:03.148200 5162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:03.148300 5162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:46:03.153788 5162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:03.154901 5162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:03.156078 5162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:03.165848 5172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:03.165848 5167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:03.165884 5168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:03.166081 5162 server_base.cc:1034] running on GCE node
I20260811 04:46:03.166565 5162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:03.167145 5162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:03.168363 5162 hybrid_clock.cc:648] HybridClock initialized: now 1786423563168333 us; error 40 us; skew 500 ppm
Aug 11 04:46:03 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423563, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:03.171231 5162 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:46:03.172153 5162 webserver.cc:460] Webserver started at http://127.22.144.130:33223/ using document root <none> and password file <none>
I20260811 04:46:03.172750 5162 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:03.172865 5162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:03.173139 5162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:03.176156 5162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "3b39f25aece44c96a43d938c6a99c775"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "6f1d5582571aa6e6f1c7739344cb0aa8"
server_key_iv: "d3ef70265fbec15f1e4dce76afe513ff"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.176774 5162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "3b39f25aece44c96a43d938c6a99c775"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "6f1d5582571aa6e6f1c7739344cb0aa8"
server_key_iv: "d3ef70265fbec15f1e4dce76afe513ff"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.180687 5162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:46:03.183321 5182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.184662 5162 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:46:03.184804 5162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "3b39f25aece44c96a43d938c6a99c775"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "6f1d5582571aa6e6f1c7739344cb0aa8"
server_key_iv: "d3ef70265fbec15f1e4dce76afe513ff"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.184926 5162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:03.202533 5162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:03.206172 5162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:03.206385 5162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:03.207027 5162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:03.207893 5162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:03.207973 5162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.208067 5162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:03.208119 5162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.217676 5162 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:38005
I20260811 04:46:03.217700 5314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:38005 every 8 connection(s)
I20260811 04:46:03.218633 5162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:46:03.226470 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 5162
I20260811 04:46:03.226591 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:46:03.226876 23106 external_mini_cluster.cc:1428] Setting key 45377fa87d308cccdbed59b96ee12082
Aug 11 04:46:03 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423563, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:03.234414 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.220457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:43347 (local address 127.22.144.190:41843)
0811 04:46:03.220607 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.220612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.221712 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 04:46:03.223368 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.223372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.223435 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.223557 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:03.224853 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.225477 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.226913 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.227256 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.227376 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.230350 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:03.230357 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:03.230363 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:03.230399 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:03.231943 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.232628 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.232630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.232631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.232710 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.233145 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.233147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.233148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.233329 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:03.233406 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.234179 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.234293 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:46:03.235800 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.220894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:43347)
0811 04:46:03.221472 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.221593 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.223116 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.223749 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.223756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.224155 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:03.224682 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.224697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.225676 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.225682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.226769 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.226781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.227467 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.227473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.227692 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.228347 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:03.228373 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:03.230174 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:03.232068 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:03.232077 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:03.232103 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:03.232497 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:03.232808 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:03.232814 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:03.232819 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:03.233035 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:03.233451 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:03.233462 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:03.233854 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.234146 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.234536 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":118,"threads_started":1}
I20260811 04:46:03.237345 5315 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41843
I20260811 04:46:03.237658 5315 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:46:03.238305 5315 heartbeater.cc:511] Master 127.22.144.190:41843 requested a full tablet report, sending...
I20260811 04:46:03.239494 4916 ts_manager.cc:194] Registered new tserver with Master: 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005)
I20260811 04:46:03.240116 4916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:43347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:46:03.259048 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41843
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:03.376756 5320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:03.377041 5320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:03.377087 5320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:03.381302 5320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:03.381389 5320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:03.381451 5320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:46:03.387203 5320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:03.388430 5320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:03.389544 5320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:03.399736 5328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:03.399750 5326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:03.399744 5331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:03.400142 5320 server_base.cc:1034] running on GCE node
I20260811 04:46:03.400668 5320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:03.401371 5320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:03.402591 5320 hybrid_clock.cc:648] HybridClock initialized: now 1786423563402569 us; error 33 us; skew 500 ppm
Aug 11 04:46:03 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423563, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:03.405460 5320 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:46:03.406653 5320 webserver.cc:460] Webserver started at http://127.22.144.131:44871/ using document root <none> and password file <none>
I20260811 04:46:03.407261 5320 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:03.407359 5320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:03.407598 5320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:03.409811 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "867fb1553d0194710a2abe739cf5df64"
server_key_iv: "be3fc8ca69391b83789fbd0e8268fbd8"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.410395 5320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "867fb1553d0194710a2abe739cf5df64"
server_key_iv: "be3fc8ca69391b83789fbd0e8268fbd8"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.414381 5320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:46:03.417232 5339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.418713 5320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:46:03.418870 5320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "867fb1553d0194710a2abe739cf5df64"
server_key_iv: "be3fc8ca69391b83789fbd0e8268fbd8"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.419027 5320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:03.439420 5320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:03.442776 5320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:03.443004 5320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:03.443576 5320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:03.444365 5320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:03.444410 5320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.444451 5320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:03.444473 5320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.453588 5320 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:39221
I20260811 04:46:03.453608 5473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:39221 every 8 connection(s)
I20260811 04:46:03.454660 5320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:46:03.457012 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 5320
I20260811 04:46:03.457113 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:46:03.457347 23106 external_mini_cluster.cc:1428] Setting key ac559b7f172bbe5b20009459b6dff54e
Aug 11 04:46:03 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423563, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:03.468477 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.456462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:36015 (local address 127.22.144.190:41843)
0811 04:46:03.456598 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.456609 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.457524 (+ 915us) server_negotiation.cc:407] Connection header received
0811 04:46:03.458962 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.458966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.459026 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.459139 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:03.460349 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.460891 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.462125 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.462365 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.462483 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.464972 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:03.464980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:03.464983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:03.465008 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:03.466386 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.466937 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.466940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.466942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.467029 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.467376 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.467381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.467383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.467570 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:03.467685 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.468189 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.468314 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 04:46:03.469286 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.456815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:36015)
0811 04:46:03.457303 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.457357 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.458734 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.459321 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.459329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.459690 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:03.460190 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.460205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.461021 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.461027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.461989 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.461999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.462523 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.462527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.462701 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.463321 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:03.463339 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:03.464799 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:03.466509 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:03.466515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:03.466531 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:03.466824 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:03.467131 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:03.467134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:03.467136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:03.467276 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:03.467698 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:03.467706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:03.467950 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.468148 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.468473 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":146,"threads_started":1}
I20260811 04:46:03.470641 5475 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41843
I20260811 04:46:03.470920 5475 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:03.471571 5475 heartbeater.cc:511] Master 127.22.144.190:41843 requested a full tablet report, sending...
I20260811 04:46:03.472584 4916 ts_manager.cc:194] Registered new tserver with Master: a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
I20260811 04:46:03.473191 4916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:36015
I20260811 04:46:03.483219 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:46:03.492918 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.485280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.22.144.190:41843)
0811 04:46:03.485400 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.485405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.485620 (+ 215us) server_negotiation.cc:407] Connection header received
0811 04:46:03.485763 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.485766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.485806 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.485873 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:03.486634 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.487289 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.488185 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.488393 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.488471 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.489948 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:03.489951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:03.489953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:03.489977 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:03.491481 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.491882 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.491884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.491885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.491958 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:03.492185 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:03.492187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:03.492188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:03.492399 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:03.492498 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.492702 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.492801 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 04:46:03.495860 4916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:46:03.498998 4916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:46:03.514252 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.508661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54960 (local address 127.22.144.129:45537)
0811 04:46:03.509147 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.509168 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.509542 (+ 374us) server_negotiation.cc:407] Connection header received
0811 04:46:03.509624 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.509627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.509797 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.509943 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:03.510352 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.511557 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.512818 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.513562 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.513688 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.513827 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.513991 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.514089 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":115,"threads_started":1}
I20260811 04:46:03.514241 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.508169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:45537 (local address 127.0.0.1:54960)
0811 04:46:03.509295 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.509307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.509384 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.510079 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.510095 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.510119 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:03.510240 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.510246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.511713 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.511716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.512679 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.512690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.513690 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.513694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.513798 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.513836 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.513950 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.514090 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"mutex_wait_us":58,"thread_start_us":92,"threads_started":1}
I20260811 04:46:03.514570 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.507953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35382 (local address 127.22.144.131:39221)
0811 04:46:03.509259 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.509269 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.509315 (+ 46us) server_negotiation.cc:407] Connection header received
0811 04:46:03.509399 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.509406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.509663 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.509844 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:03.510183 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.511123 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.512890 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.513601 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.513735 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.513861 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.514319 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.514440 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1176,"thread_start_us":176,"threads_started":1}
I20260811 04:46:03.515158 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.507891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38005 (local address 127.0.0.1:36200)
0811 04:46:03.509215 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.509230 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.509344 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.510041 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.510057 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.510075 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:03.510253 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.510262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.511713 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.511716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.513036 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.513045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.514307 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.514763 (+ 456us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.514857 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.514884 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.514967 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.515036 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":179,"thread_start_us":106,"threads_started":1}
I20260811 04:46:03.515292 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.508663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.22.144.130:38005)
0811 04:46:03.509073 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.509085 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.509340 (+ 255us) server_negotiation.cc:407] Connection header received
0811 04:46:03.509507 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.509513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.509689 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.509846 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:03.510404 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.511557 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.513174 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.514009 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.514133 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.514244 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.515000 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.515103 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":116,"threads_started":1}
I20260811 04:46:03.515393 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.507822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39221 (local address 127.0.0.1:35382)
0811 04:46:03.508721 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.508775 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.508901 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.509850 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.509860 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.509891 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:03.510052 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.510061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.511303 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.511306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.512747 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.512758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.513910 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.513914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.514004 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.514028 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.514113 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.514190 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260811 04:46:03.517663 5072 tablet_service.cc:1467] Processing CreateTablet for tablet d89468ce19e44a61be1244da6320402b (DEFAULT_TABLE table=test-table [id=096018f9e81c4d938af4bef9465a3d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:03.518237 5395 tablet_service.cc:1467] Processing CreateTablet for tablet d89468ce19e44a61be1244da6320402b (DEFAULT_TABLE table=test-table [id=096018f9e81c4d938af4bef9465a3d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:03.518395 5238 tablet_service.cc:1467] Processing CreateTablet for tablet d89468ce19e44a61be1244da6320402b (DEFAULT_TABLE table=test-table [id=096018f9e81c4d938af4bef9465a3d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:03.518842 5072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89468ce19e44a61be1244da6320402b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:03.519415 5238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89468ce19e44a61be1244da6320402b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:03.519407 5395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89468ce19e44a61be1244da6320402b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:03.526350 5493 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap starting.
I20260811 04:46:03.527009 5495 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Bootstrap starting.
I20260811 04:46:03.527693 5494 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Bootstrap starting.
I20260811 04:46:03.529012 5495 tablet_bootstrap.cc:654] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:03.530333 5494 tablet_bootstrap.cc:654] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:03.530349 5495 log.cc:824] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:03.531358 5494 log.cc:824] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:03.532730 5495 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: No bootstrap required, opened a new log
I20260811 04:46:03.532847 5493 tablet_bootstrap.cc:654] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:03.532939 5495 ts_tablet_manager.cc:1397] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 04:46:03.533535 5494 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: No bootstrap required, opened a new log
I20260811 04:46:03.533749 5494 ts_tablet_manager.cc:1397] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:46:03.533733 5493 log.cc:824] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:03.535835 5493 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: No bootstrap required, opened a new log
I20260811 04:46:03.536010 5493 ts_tablet_manager.cc:1397] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 04:46:03.536878 5495 raft_consensus.cc:348] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.537148 5495 raft_consensus.cc:374] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:03.537199 5495 raft_consensus.cc:729] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96d37f0ad2646a18598ca3d1c0732a1, State: Initialized, Role: FOLLOWER
I20260811 04:46:03.537364 5494 raft_consensus.cc:348] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.537719 5494 raft_consensus.cc:374] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:03.537806 5494 raft_consensus.cc:729] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b39f25aece44c96a43d938c6a99c775, State: Initialized, Role: FOLLOWER
I20260811 04:46:03.538301 5495 consensus_queue.cc:260] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.538314 5494 consensus_queue.cc:260] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.539098 5475 heartbeater.cc:503] Master 127.22.144.190:41843 was elected leader, sending a full tablet report...
I20260811 04:46:03.539399 5495 ts_tablet_manager.cc:1428] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 04:46:03.539459 5494 ts_tablet_manager.cc:1428] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 04:46:03.539824 5315 heartbeater.cc:503] Master 127.22.144.190:41843 was elected leader, sending a full tablet report...
I20260811 04:46:03.540449 5493 raft_consensus.cc:348] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.540753 5493 raft_consensus.cc:374] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:03.540822 5493 raft_consensus.cc:729] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a6fc5ea5304a39b43c0b1e8c0b3b84, State: Initialized, Role: FOLLOWER
I20260811 04:46:03.541236 5493 consensus_queue.cc:260] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.542109 5493 ts_tablet_manager.cc:1428] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 04:46:03.542485 5154 heartbeater.cc:503] Master 127.22.144.190:41843 was elected leader, sending a full tablet report...
I20260811 04:46:03.631915 5504 raft_consensus.cc:482] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:03.632166 5504 raft_consensus.cc:504] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.633387 5504 leader_election.cc:288] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221), 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005)
I20260811 04:46:03.637535 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.633916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:37743 (local address 127.22.144.130:38005)
0811 04:46:03.634149 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.634154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.634323 (+ 169us) server_negotiation.cc:407] Connection header received
0811 04:46:03.634478 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.634481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.634532 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.634614 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:03.634919 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.635412 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.636516 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.636922 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.637056 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.637122 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.637309 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.637376 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 04:46:03.637544 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.633770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38005 (local address 127.22.144.129:37743)
0811 04:46:03.634229 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.634240 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.634362 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.634649 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.634652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.634670 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:03.634799 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.634804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.635535 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.635541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.636384 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.636394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.637053 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.637057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.637162 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.637187 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.637292 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.637363 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":131,"threads_started":1}
I20260811 04:46:03.637681 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.633720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:40313 (local address 127.22.144.131:39221)
0811 04:46:03.633906 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.633914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.634216 (+ 302us) server_negotiation.cc:407] Connection header received
0811 04:46:03.634417 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.634421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.634476 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.634573 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:03.634878 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.635434 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.636561 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.637010 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.637152 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.637214 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.637419 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.637530 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 04:46:03.638000 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.633753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39221 (local address 127.22.144.129:40313)
0811 04:46:03.634105 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.634119 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:03.634283 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:03.634588 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:03.634592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:03.634610 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:03.634751 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.634757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.635568 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.635573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.636409 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:03.636418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.637145 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.637149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:03.637273 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.637304 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 04:46:03.637401 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 04:46:03.637478 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":148,"threads_started":1}
I20260811 04:46:03.638347 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89468ce19e44a61be1244da6320402b" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b39f25aece44c96a43d938c6a99c775" is_pre_election: true
I20260811 04:46:03.638746 5260 raft_consensus.cc:2454] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 0.
I20260811 04:46:03.638715 5417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89468ce19e44a61be1244da6320402b" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" is_pre_election: true
I20260811 04:46:03.639034 5417 raft_consensus.cc:2454] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 0.
I20260811 04:46:03.639349 5016 leader_election.cc:302] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b39f25aece44c96a43d938c6a99c775, f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters:
I20260811 04:46:03.639714 5504 raft_consensus.cc:2781] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:46:03.639808 5504 raft_consensus.cc:482] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:46:03.639853 5504 raft_consensus.cc:3037] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:03.640918 5504 raft_consensus.cc:504] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.641256 5504 leader_election.cc:288] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 election: Requested vote from peers a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221), 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005)
I20260811 04:46:03.641667 5417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89468ce19e44a61be1244da6320402b" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
I20260811 04:46:03.641683 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89468ce19e44a61be1244da6320402b" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b39f25aece44c96a43d938c6a99c775"
I20260811 04:46:03.641798 5417 raft_consensus.cc:3037] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:03.641829 5260 raft_consensus.cc:3037] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:03.642869 5417 raft_consensus.cc:2454] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 1.
I20260811 04:46:03.642906 5260 raft_consensus.cc:2454] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 1.
I20260811 04:46:03.643183 5015 leader_election.cc:302] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96d37f0ad2646a18598ca3d1c0732a1, f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters:
I20260811 04:46:03.643401 5504 raft_consensus.cc:2781] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:03.643600 5504 raft_consensus.cc:686] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 LEADER]: Becoming Leader. State: Replica: f5a6fc5ea5304a39b43c0b1e8c0b3b84, State: Running, Role: LEADER
I20260811 04:46:03.643923 5504 consensus_queue.cc:237] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:03.647428 4914 catalog_manager.cc:5495] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129). New cstate: current_term: 1 leader_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } health_report { overall_health: HEALTHY } } }
I20260811 04:46:03.667990 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.664258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54966 (local address 127.22.144.129:45537)
0811 04:46:03.664384 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:03.664389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:03.664403 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:46:03.664509 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:03.664513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:03.664568 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:03.664748 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:46:03.665013 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.665591 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.666471 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:03.666701 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:03.666818 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:03.667157 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:46:03.667560 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:46:03.667708 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:03.667776 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 04:46:03.667839 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 04:46:03.674046 5417 raft_consensus.cc:1261] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Refusing update from remote peer f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:03.674091 5260 raft_consensus.cc:1261] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Refusing update from remote peer f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:03.674862 5507 consensus_queue.cc:1035] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:03.675062 5504 consensus_queue.cc:1035] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:03.686141 5512 mvcc.cc:205] Tried to move back new op lower bound from 7317190916801290240 to 7317190916688216064. Current Snapshot: MvccSnapshot[applied={T|T < 7317190916801290240 or (T in {7317190916801290240})}]
I20260811 04:46:03.694885 4916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:03.695099 4916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:03.698115 4916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Sending DeleteTablet for 3 replicas of tablet d89468ce19e44a61be1244da6320402b
I20260811 04:46:03.699090 5072 tablet_service.cc:1514] Processing DeleteTablet for tablet d89468ce19e44a61be1244da6320402b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:03 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54960
I20260811 04:46:03.699226 5395 tablet_service.cc:1514] Processing DeleteTablet for tablet d89468ce19e44a61be1244da6320402b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:03 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:35382
I20260811 04:46:03.699260 5238 tablet_service.cc:1514] Processing DeleteTablet for tablet d89468ce19e44a61be1244da6320402b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:03 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:36200
I20260811 04:46:03.699751 5525 tablet_replica.cc:330] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: stopping tablet replica
I20260811 04:46:03.700049 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 4991
I20260811 04:46:03.703401 5524 tablet_replica.cc:330] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: stopping tablet replica
I20260811 04:46:03.703780 5524 raft_consensus.cc:2229] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:03.704030 5525 raft_consensus.cc:2229] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:03.704136 5524 raft_consensus.cc:2258] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:03.704429 5525 raft_consensus.cc:2258] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:03.705960 5524 ts_tablet_manager.cc:1905] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:03.705988 5525 ts_tablet_manager.cc:1905] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:03.708892 5524 ts_tablet_manager.cc:1918] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:03.708892 5525 ts_tablet_manager.cc:1918] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:03.709008 5525 log.cc:1196] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/d89468ce19e44a61be1244da6320402b
I20260811 04:46:03.709008 5524 log.cc:1196] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/d89468ce19e44a61be1244da6320402b
I20260811 04:46:03.709365 5524 ts_tablet_manager.cc:1939] T d89468ce19e44a61be1244da6320402b P 3b39f25aece44c96a43d938c6a99c775: Deleting consensus metadata
I20260811 04:46:03.709365 5525 ts_tablet_manager.cc:1939] T d89468ce19e44a61be1244da6320402b P a96d37f0ad2646a18598ca3d1c0732a1: Deleting consensus metadata
I20260811 04:46:03.710839 4901 catalog_manager.cc:4841] TS 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): tablet d89468ce19e44a61be1244da6320402b (table test-table [id=096018f9e81c4d938af4bef9465a3d97]) successfully deleted
W20260811 04:46:03.711027 4898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:45537 (error 108)
I20260811 04:46:03.711182 4899 catalog_manager.cc:4841] TS a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): tablet d89468ce19e44a61be1244da6320402b (table test-table [id=096018f9e81c4d938af4bef9465a3d97]) successfully deleted
I20260811 04:46:03.711376 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 5162
I20260811 04:46:03.713948 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:03.711904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:45537 (local address 127.0.0.1:54970)
0811 04:46:03.713713 (+ 1809us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:03.713814 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1672}
W20260811 04:46:03.714481 4898 catalog_manager.cc:4584] TS f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129:45537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d89468ce19e44a61be1244da6320402b: Network error: Client connection negotiation failed: client connection to 127.22.144.129:45537: connect: Connection refused (error 111)
I20260811 04:46:03.721415 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 5320
I20260811 04:46:03.732362 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 4877
I20260811 04:46:03.741586 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41843
--webserver_interface=127.22.144.190
--webserver_port=44837
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:03.858448 5528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:03.858749 5528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:03.858793 5528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:03.863196 5528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:46:03.863268 5528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:03.863288 5528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:03.863306 5528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:46:03.863322 5528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:46:03.868856 5528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=44837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:03.870149 5528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:03.871248 5528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:03.886390 5537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:03.886418 5534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:03.886439 5535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:03.887048 5528 server_base.cc:1034] running on GCE node
I20260811 04:46:03.887820 5528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:03.889155 5528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:03.890358 5528 hybrid_clock.cc:648] HybridClock initialized: now 1786423563890317 us; error 65 us; skew 500 ppm
Aug 11 04:46:03 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423563, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:03.893460 5528 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:46:03.894698 5528 webserver.cc:460] Webserver started at http://127.22.144.190:44837/ using document root <none> and password file <none>
I20260811 04:46:03.895340 5528 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:03.895457 5528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:03.899761 5528 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:46:03.902199 5548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:03.903462 5528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:03.903620 5528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "733a1b1464144dd29e36fb84388d164b"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "c7e376fe71471d6995f2a04729be3c8b"
server_key_iv: "7e3c467c9d0532474631ddf73ae8000d"
server_key_version: "encryptionkey@0"
I20260811 04:46:03.904310 5528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:03.918395 5528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:03.922950 5528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:03.923203 5528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:03.931063 5528 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:41843
I20260811 04:46:03.931367 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:41843 every 8 connection(s)
I20260811 04:46:03.932320 5528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:46:03.936990 5613 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:46:03.938859 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap starting.
I20260811 04:46:03.939504 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 5528
I20260811 04:46:03.939987 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:45537
--local_ip_for_outbound_sockets=127.22.144.129
--tserver_master_addrs=127.22.144.190:41843
--webserver_port=44033
--webserver_interface=127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:46:03.963462 5613 log.cc:824] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:03.974884 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:46:03.975342 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap complete.
I20260811 04:46:03.981130 5613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:03.981675 5613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733a1b1464144dd29e36fb84388d164b, State: Initialized, Role: FOLLOWER
I20260811 04:46:03.982247 5613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:03.982394 5613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:46:03.982468 5613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:46:03.982563 5613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 1 FOLLOWER]: Advancing to term 2
I20260811 04:46:03.985105 5613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:03.985522 5613 leader_election.cc:302] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 733a1b1464144dd29e36fb84388d164b; no voters:
I20260811 04:46:03.985924 5613 leader_election.cc:288] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 04:46:03.986091 5617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 04:46:03.986369 5617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 LEADER]: Becoming Leader. State: Replica: 733a1b1464144dd29e36fb84388d164b, State: Running, Role: LEADER
I20260811 04:46:03.986740 5617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:03.987313 5613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:46:03.987767 5619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 733a1b1464144dd29e36fb84388d164b. Latest consensus state: current_term: 2 leader_uuid: "733a1b1464144dd29e36fb84388d164b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } } }
I20260811 04:46:03.987915 5619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:03.988313 5618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "733a1b1464144dd29e36fb84388d164b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } } }
I20260811 04:46:03.988435 5618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:03.992839 5634 catalog_manager.cc:1259] Loaded cluster ID: 60cb661fff0d4332879ef0767ff51757
I20260811 04:46:03.992908 5634 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: loading cluster ID for follower catalog manager: success
W20260811 04:46:03.993916 5634 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260811 04:46:03.993992 5634 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260811 04:46:03.994089 5625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:46:03.996574 5625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:46:03.997082 5625 catalog_manager.cc:1259] Loaded cluster ID: 60cb661fff0d4332879ef0767ff51757
I20260811 04:46:03.997141 5625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:46:03.998276 5625 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 04:46:04.003201 5625 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 04:46:04.008800 5625 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:46:04.009637 5625 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Loaded TSK: 0
I20260811 04:46:04.010879 5625 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 04:46:04.074775 5615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:04.075107 5615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:04.075203 5615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:04.079460 5615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:04.079556 5615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:04.079659 5615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:46:04.085117 5615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:45537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=44033
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:04.086199 5615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:04.087339 5615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:04.097689 5642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:04.097721 5641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:04.097709 5645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:04.098296 5615 server_base.cc:1034] running on GCE node
I20260811 04:46:04.098740 5615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:04.099414 5615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:04.100646 5615 hybrid_clock.cc:648] HybridClock initialized: now 1786423564100613 us; error 43 us; skew 500 ppm
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423564, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:04.103699 5615 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:46:04.104650 5615 webserver.cc:460] Webserver started at http://127.22.144.129:44033/ using document root <none> and password file <none>
I20260811 04:46:04.105238 5615 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:04.105352 5615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:04.109984 5615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:04.112392 5652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.113909 5615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:04.114051 5615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "5e09262d707737001b430383f4a327e4"
server_key_iv: "fc6847f6e5e2ccffeba2a69d415ea6da"
server_key_version: "encryptionkey@0"
I20260811 04:46:04.114634 5615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:04.145776 5615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:04.149035 5615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:04.149271 5615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:04.149955 5615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:04.151132 5662 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 04:46:04.155440 5615 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 04:46:04.155534 5615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 04:46:04.155642 5615 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 04:46:04.157398 5615 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 04:46:04.157480 5615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 04:46:04.157877 5662 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap starting.
I20260811 04:46:04.168467 5615 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:45537
I20260811 04:46:04.168486 5797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:45537 every 8 connection(s)
I20260811 04:46:04.169625 5615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:46:04.178040 5662 log.cc:824] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:04.179924 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 5615
I20260811 04:46:04.180411 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:38005
--local_ip_for_outbound_sockets=127.22.144.130
--tserver_master_addrs=127.22.144.190:41843
--webserver_port=33223
--webserver_interface=127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423564, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:04.188045 5662 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:46:04.188524 5662 tablet_bootstrap.cc:492] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap complete.
I20260811 04:46:04.189615 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.171920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:44619 (local address 127.22.144.190:41843)
0811 04:46:04.172351 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.172371 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.172848 (+ 477us) server_negotiation.cc:407] Connection header received
0811 04:46:04.174088 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.174112 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.174572 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.174966 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:04.176250 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.177780 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.179523 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.180296 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.180500 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.184059 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:04.184069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:04.184088 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:04.184123 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:04.186044 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.186606 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.186611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.186615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.186737 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.187123 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.187126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.187127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.187650 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:04.187785 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.188446 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.188798 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":138,"threads_started":1}
I20260811 04:46:04.190104 5662 ts_tablet_manager.cc:1397] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.014s
I20260811 04:46:04.190901 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.171923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:44619)
0811 04:46:04.172650 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.172703 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.173591 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.175055 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.175063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.175469 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:04.176007 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.176031 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.177995 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.178001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.179387 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.179399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.180483 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.180751 (+ 268us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.181006 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.181796 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:04.181824 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:04.183819 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:04.186175 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.186181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.186195 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.186456 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.186839 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.186844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.186846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.186983 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.187774 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:04.187780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:04.188036 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.189599 (+ 1563us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.190241 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":158,"threads_started":1}
I20260811 04:46:04.193151 5799 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41843
I20260811 04:46:04.193415 5799 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:04.193923 5799 heartbeater.cc:511] Master 127.22.144.190:41843 requested a full tablet report, sending...
I20260811 04:46:04.194042 5662 raft_consensus.cc:348] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.194715 5662 raft_consensus.cc:729] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a6fc5ea5304a39b43c0b1e8c0b3b84, State: Initialized, Role: FOLLOWER
I20260811 04:46:04.195293 5662 consensus_queue.cc:260] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.197985 5662 ts_tablet_manager.cc:1428] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260811 04:46:04.198603 5571 ts_manager.cc:194] Registered new tserver with Master: f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129:45537)
I20260811 04:46:04.202067 5571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:44619
I20260811 04:46:04.204703 5799 heartbeater.cc:503] Master 127.22.144.190:41843 was elected leader, sending a full tablet report...
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423563, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:46:04.213027 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.201460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.22.144.129:45537)
0811 04:46:04.202148 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.202220 (+ 72us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.202247 (+ 27us) server_negotiation.cc:407] Connection header received
0811 04:46:04.202317 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.202323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.202524 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.202667 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:04.203401 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.204438 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.205768 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.206431 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.206556 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.209314 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:04.209322 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:04.209333 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:04.209355 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:04.210991 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.211594 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.211600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.211602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.211694 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.212009 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.212012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.212013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.212260 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:04.212374 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.212740 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.212895 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":107,"threads_started":1}
I20260811 04:46:04.213028 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.201365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:45537 (local address 127.0.0.1:54984)
0811 04:46:04.201813 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.201866 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.202010 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.202697 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.202704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.203141 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:04.203270 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.203279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.204574 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.204578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.205625 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.205637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.206556 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.206560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.206679 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.207260 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:04.207272 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:04.209174 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:04.211129 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.211139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.211146 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.211451 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.211800 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.211803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.211805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.211903 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.212373 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:04.212379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:04.212507 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.212665 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.212859 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":148,"threads_started":1}
I20260811 04:46:04.214126 5716 tablet_service.cc:1514] Processing DeleteTablet for tablet d89468ce19e44a61be1244da6320402b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:03 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54984
I20260811 04:46:04.214982 5808 tablet_replica.cc:330] stopping tablet replica
I20260811 04:46:04.215286 5808 raft_consensus.cc:2229] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:04.215548 5808 raft_consensus.cc:2258] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:04.216511 5808 ts_tablet_manager.cc:1905] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:04.220575 5808 ts_tablet_manager.cc:1918] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:04.220664 5808 log.cc:1196] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/d89468ce19e44a61be1244da6320402b
I20260811 04:46:04.220949 5808 ts_tablet_manager.cc:1939] T d89468ce19e44a61be1244da6320402b P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Deleting consensus metadata
I20260811 04:46:04.222811 5551 catalog_manager.cc:4841] TS f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129:45537): tablet d89468ce19e44a61be1244da6320402b (table test-table [id=096018f9e81c4d938af4bef9465a3d97]) successfully deleted
W20260811 04:46:04.313180 5804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:04.313436 5804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:04.313525 5804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:04.317430 5804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:04.317546 5804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:04.317649 5804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:46:04.323100 5804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:38005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=33223
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:04.324131 5804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:04.325129 5804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:04.335032 5817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:04.335009 5815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:04.335075 5804 server_base.cc:1034] running on GCE node
W20260811 04:46:04.335016 5814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:04.335625 5804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:04.336282 5804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:04.337468 5804 hybrid_clock.cc:648] HybridClock initialized: now 1786423564337429 us; error 51 us; skew 500 ppm
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423564, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:04.340258 5804 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:46:04.341132 5804 webserver.cc:460] Webserver started at http://127.22.144.130:33223/ using document root <none> and password file <none>
I20260811 04:46:04.341766 5804 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:04.341879 5804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:04.345896 5804 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:46:04.348022 5825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.349301 5804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:46:04.349440 5804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "3b39f25aece44c96a43d938c6a99c775"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "6f1d5582571aa6e6f1c7739344cb0aa8"
server_key_iv: "d3ef70265fbec15f1e4dce76afe513ff"
server_key_version: "encryptionkey@0"
I20260811 04:46:04.350122 5804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:04.362460 5804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:04.365569 5804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:04.365808 5804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:04.366443 5804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:04.367336 5804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:04.367408 5804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.367482 5804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:04.367524 5804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.377100 5804 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:38005
I20260811 04:46:04.377168 5961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:38005 every 8 connection(s)
I20260811 04:46:04.378201 5804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:46:04.380649 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 5804
I20260811 04:46:04.381181 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:39221
--local_ip_for_outbound_sockets=127.22.144.131
--tserver_master_addrs=127.22.144.190:41843
--webserver_port=44871
--webserver_interface=127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423564, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:04.395008 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.380119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:51931 (local address 127.22.144.190:41843)
0811 04:46:04.380266 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.380269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.381192 (+ 923us) server_negotiation.cc:407] Connection header received
0811 04:46:04.383450 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.383453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.383518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.383611 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:04.384925 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.385530 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.386714 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.387042 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.387158 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.390296 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:04.390303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:04.390308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:04.390349 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:04.392308 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.392859 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.392866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.392870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.392992 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.393371 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.393377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.393380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.393733 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:04.393887 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.394550 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.394738 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:46:04.396147 5964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.380452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:51931)
0811 04:46:04.380988 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.381048 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.383117 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.383800 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.383809 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.384270 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:04.384770 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.384784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.385654 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.385658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.386585 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.386597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.387216 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.387220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.387405 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.388119 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:04.388136 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:04.390084 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:04.392444 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.392450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.392463 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.392744 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.393112 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.393114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.393116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.393262 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.393911 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:04.393923 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:04.394229 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.394426 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.394752 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
I20260811 04:46:04.397640 5962 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41843
I20260811 04:46:04.397920 5962 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:04.398522 5962 heartbeater.cc:511] Master 127.22.144.190:41843 requested a full tablet report, sending...
I20260811 04:46:04.399706 5571 ts_manager.cc:194] Registered new tserver with Master: 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005)
I20260811 04:46:04.400568 5571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:51931
W20260811 04:46:04.506903 5966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:04.507179 5966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:04.507223 5966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:04.511507 5966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:04.511591 5966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:04.511684 5966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:46:04.517166 5966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:39221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=44871
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:04.518455 5966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:04.519650 5966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:04.529587 5972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:04.529757 5966 server_base.cc:1034] running on GCE node
W20260811 04:46:04.529592 5974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:04.529675 5971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:04.530310 5966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:04.531034 5966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:04.532228 5966 hybrid_clock.cc:648] HybridClock initialized: now 1786423564532204 us; error 44 us; skew 500 ppm
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423564, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:04.535135 5966 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:46:04.536015 5966 webserver.cc:460] Webserver started at http://127.22.144.131:44871/ using document root <none> and password file <none>
I20260811 04:46:04.536607 5966 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:04.536722 5966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:04.540697 5966 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:04.542843 5983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.544034 5966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 04:46:04.544188 5966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "867fb1553d0194710a2abe739cf5df64"
server_key_iv: "be3fc8ca69391b83789fbd0e8268fbd8"
server_key_version: "encryptionkey@0"
I20260811 04:46:04.544742 5966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:04.559974 5966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:04.563498 5966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:04.563724 5966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:04.564345 5966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:04.565189 5966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:04.565263 5966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.565337 5966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:04.565380 5966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:04.574227 5966 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:39221
I20260811 04:46:04.574258 6112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:39221 every 8 connection(s)
I20260811 04:46:04.575234 5966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:46:04.580904 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 5966
Aug 11 04:46:04 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423564, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:04.590425 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.581689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.22.144.190:41843)
0811 04:46:04.581973 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.581978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.582142 (+ 164us) server_negotiation.cc:407] Connection header received
0811 04:46:04.582258 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.582262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.582311 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.582418 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:04.583108 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.583727 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.584668 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.585056 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.585175 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.586552 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:04.586559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:04.586561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:04.586588 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:04.588624 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.589075 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.589078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.589079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.589202 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.589542 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.589545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.589547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.589734 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:04.589957 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.590118 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.590246 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
I20260811 04:46:04.591419 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.577107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:35505 (local address 127.22.144.190:41843)
0811 04:46:04.577281 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.577285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.578197 (+ 912us) server_negotiation.cc:407] Connection header received
0811 04:46:04.579633 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.579636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.579686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.579766 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:04.581455 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.582323 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.583833 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.584156 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.584277 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.587103 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:04.587107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:04.587110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:04.587156 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:04.589021 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.589713 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.589717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.589719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.589808 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:04.590161 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:04.590165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:04.590166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:04.590378 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:04.590469 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.591161 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.591270 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 04:46:04.592253 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.577453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:35505)
0811 04:46:04.578002 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.578072 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.579422 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.580007 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.580015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.580414 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:04.581226 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.581250 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.582566 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.582569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.583695 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.583707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.584324 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.584331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.584559 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.585215 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:04.585238 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:04.586928 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:04.589165 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.589171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.589207 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.589607 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.589903 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:04.589907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:04.589908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:04.590055 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:04.590535 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:04.590543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:04.590850 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.591068 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.591387 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":113,"threads_started":1}
I20260811 04:46:04.593340 6115 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41843
I20260811 04:46:04.593613 6115 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:04.594101 6115 heartbeater.cc:511] Master 127.22.144.190:41843 requested a full tablet report, sending...
I20260811 04:46:04.595100 5571 ts_manager.cc:194] Registered new tserver with Master: a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
I20260811 04:46:04.595693 5571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:35505
I20260811 04:46:04.604215 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:46:04.608633 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.604885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.22.144.190:41843)
0811 04:46:04.605051 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.605054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.605172 (+ 118us) server_negotiation.cc:407] Connection header received
0811 04:46:04.605306 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.605308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.605405 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.605542 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:46:04.605770 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.606279 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.607150 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.607352 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.607427 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.607769 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:46:04.608234 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:46:04.608327 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.608418 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.608464 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 04:46:04.609352 5571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:46:04.613061 5571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:46:04.621268 5716 tablet_service.cc:1467] Processing CreateTablet for tablet 2d42ba938b6c43ec89295d3c4a371bc3 (DEFAULT_TABLE table=test-table [id=84ce3cb7bbd943d29cd144c3868fd388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:04.622004 5716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d42ba938b6c43ec89295d3c4a371bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:04.624977 5662 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap starting.
I20260811 04:46:04.625748 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.620047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38005 (local address 127.0.0.1:36206)
0811 04:46:04.620333 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.620350 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.620431 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.621455 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.621459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.621505 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:04.621621 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.621629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.622915 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.622919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.624267 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.624280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.625277 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.625282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.625374 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.625434 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.625491 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.625607 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":70}
I20260811 04:46:04.625937 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.620271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39221 (local address 127.0.0.1:35394)
0811 04:46:04.620627 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.620642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.620727 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.621668 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.621674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.621710 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:04.621819 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.621825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.623329 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.623332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.624360 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.624388 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.625530 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.625534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.625630 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.625652 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.625731 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.625804 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260811 04:46:04.626016 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.620813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.22.144.131:39221)
0811 04:46:04.621156 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.621168 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.621193 (+ 25us) server_negotiation.cc:407] Connection header received
0811 04:46:04.621270 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.621275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.621501 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.621972 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:04.622148 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.623186 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.624516 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.625379 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.625539 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.625681 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.625773 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.625867 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":102,"threads_started":1}
I20260811 04:46:04.626143 5662 tablet_bootstrap.cc:654] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:04.626720 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.620253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.22.144.130:38005)
0811 04:46:04.620994 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.621006 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.621031 (+ 25us) server_negotiation.cc:407] Connection header received
0811 04:46:04.621123 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.621128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.621300 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.621561 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:04.621724 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.622768 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.624401 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.625138 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.625937 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.626360 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.626414 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.626525 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":133,"threads_started":1}
I20260811 04:46:04.627754 5662 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: No bootstrap required, opened a new log
I20260811 04:46:04.627849 5662 ts_tablet_manager.cc:1397] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:04.628306 5662 raft_consensus.cc:348] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.628428 5662 raft_consensus.cc:374] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:04.628464 5662 raft_consensus.cc:729] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a6fc5ea5304a39b43c0b1e8c0b3b84, State: Initialized, Role: FOLLOWER
I20260811 04:46:04.628630 5662 consensus_queue.cc:260] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.628890 5662 ts_tablet_manager.cc:1428] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:46:04.629230 6040 tablet_service.cc:1467] Processing CreateTablet for tablet 2d42ba938b6c43ec89295d3c4a371bc3 (DEFAULT_TABLE table=test-table [id=84ce3cb7bbd943d29cd144c3868fd388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:04.630228 6040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d42ba938b6c43ec89295d3c4a371bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:04.630143 5883 tablet_service.cc:1467] Processing CreateTablet for tablet 2d42ba938b6c43ec89295d3c4a371bc3 (DEFAULT_TABLE table=test-table [id=84ce3cb7bbd943d29cd144c3868fd388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:04.631127 5883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d42ba938b6c43ec89295d3c4a371bc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:04.637168 6129 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Bootstrap starting.
I20260811 04:46:04.637665 6130 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Bootstrap starting.
I20260811 04:46:04.639240 6129 tablet_bootstrap.cc:654] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:04.639914 6130 tablet_bootstrap.cc:654] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:04.640378 6129 log.cc:824] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:04.640666 6130 log.cc:824] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:04.642709 6129 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: No bootstrap required, opened a new log
I20260811 04:46:04.642704 6130 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: No bootstrap required, opened a new log
I20260811 04:46:04.642905 6129 ts_tablet_manager.cc:1397] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:46:04.642905 6130 ts_tablet_manager.cc:1397] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 04:46:04.647095 6130 raft_consensus.cc:348] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.647113 6129 raft_consensus.cc:348] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.647395 6130 raft_consensus.cc:374] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:04.647395 6129 raft_consensus.cc:374] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:04.647457 6130 raft_consensus.cc:729] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b39f25aece44c96a43d938c6a99c775, State: Initialized, Role: FOLLOWER
I20260811 04:46:04.647467 6129 raft_consensus.cc:729] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96d37f0ad2646a18598ca3d1c0732a1, State: Initialized, Role: FOLLOWER
I20260811 04:46:04.647954 6130 consensus_queue.cc:260] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.647954 6129 consensus_queue.cc:260] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.648701 6115 heartbeater.cc:503] Master 127.22.144.190:41843 was elected leader, sending a full tablet report...
I20260811 04:46:04.649034 6130 ts_tablet_manager.cc:1428] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 04:46:04.649034 6129 ts_tablet_manager.cc:1428] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:46:04.649711 5962 heartbeater.cc:503] Master 127.22.144.190:41843 was elected leader, sending a full tablet report...
W20260811 04:46:04.670826 5800 tablet.cc:2369] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:46:04.750420 5805 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:04.750759 5805 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.751948 5805 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
I20260811 04:46:04.757484 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.752234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39221 (local address 127.22.144.129:60007)
0811 04:46:04.752576 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.752591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.752730 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.753070 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.753073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.753105 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:04.753301 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.753307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.754518 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.754522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.755946 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.755955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.756948 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.756954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.757054 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.757125 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.757210 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.757285 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":97,"spinlock_wait_cycles":6016,"thread_start_us":72,"threads_started":1}
I20260811 04:46:04.757531 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.752383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:60007 (local address 127.22.144.131:39221)
0811 04:46:04.752518 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.752523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.752684 (+ 161us) server_negotiation.cc:407] Connection header received
0811 04:46:04.752873 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.752877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.752934 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.753042 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:04.753458 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.754388 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.756093 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.756798 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.756946 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.757020 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.757262 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.757363 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:46:04.757616 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.752386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:32825 (local address 127.22.144.130:38005)
0811 04:46:04.752528 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.752533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.752745 (+ 212us) server_negotiation.cc:407] Connection header received
0811 04:46:04.752878 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.752881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.752934 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.753042 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:04.753443 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.754386 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.756093 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.756797 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.756948 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.757006 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.757370 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.757450 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:46:04.758164 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.752234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38005 (local address 127.22.144.129:32825)
0811 04:46:04.752679 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.752692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:04.752769 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:04.753070 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:04.753073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:04.753096 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:04.753291 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.753301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.754518 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.754522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.755942 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:04.755955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.757086 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.757090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:04.757209 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.757248 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 04:46:04.757335 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 04:46:04.757425 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":183,"spinlock_wait_cycles":768,"thread_start_us":124,"threads_started":1}
I20260811 04:46:04.758148 6062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d42ba938b6c43ec89295d3c4a371bc3" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" is_pre_election: true
I20260811 04:46:04.758512 6062 raft_consensus.cc:2454] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 0.
I20260811 04:46:04.758978 5904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d42ba938b6c43ec89295d3c4a371bc3" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b39f25aece44c96a43d938c6a99c775" is_pre_election: true
I20260811 04:46:04.759032 5656 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96d37f0ad2646a18598ca3d1c0732a1, f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters:
I20260811 04:46:04.759294 5904 raft_consensus.cc:2454] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 0.
I20260811 04:46:04.759359 5805 raft_consensus.cc:2781] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:46:04.759491 5805 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:46:04.759557 5805 raft_consensus.cc:3037] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:04.760782 5805 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.761245 5805 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 election: Requested vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
I20260811 04:46:04.761660 5904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d42ba938b6c43ec89295d3c4a371bc3" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b39f25aece44c96a43d938c6a99c775"
I20260811 04:46:04.761694 6062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d42ba938b6c43ec89295d3c4a371bc3" candidate_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
I20260811 04:46:04.761770 5904 raft_consensus.cc:3037] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:04.761857 6062 raft_consensus.cc:3037] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:04.762965 5904 raft_consensus.cc:2454] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 1.
I20260811 04:46:04.763077 6062 raft_consensus.cc:2454] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5a6fc5ea5304a39b43c0b1e8c0b3b84 in term 1.
I20260811 04:46:04.763288 5657 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b39f25aece44c96a43d938c6a99c775, f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters:
I20260811 04:46:04.763486 5805 raft_consensus.cc:2781] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:04.763756 5805 raft_consensus.cc:686] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 LEADER]: Becoming Leader. State: Replica: f5a6fc5ea5304a39b43c0b1e8c0b3b84, State: Running, Role: LEADER
I20260811 04:46:04.764055 5805 consensus_queue.cc:237] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:04.767050 5569 catalog_manager.cc:5495] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129). New cstate: current_term: 1 leader_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } health_report { overall_health: HEALTHY } } }
I20260811 04:46:04.790056 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.785881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54996 (local address 127.22.144.129:45537)
0811 04:46:04.786262 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:04.786269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:04.786284 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:46:04.786364 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:04.786367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:04.786426 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:04.786545 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:46:04.786775 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.787405 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.788287 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:04.788515 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:04.788668 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:04.789077 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:46:04.789596 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:46:04.789694 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:04.789776 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 04:46:04.789830 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":209,"threads_started":1}
I20260811 04:46:04.794440 5904 raft_consensus.cc:1261] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Refusing update from remote peer f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:04.794438 6062 raft_consensus.cc:1261] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Refusing update from remote peer f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:04.795310 5805 consensus_queue.cc:1035] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:04.795573 6141 consensus_queue.cc:1035] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:04.804037 6146 mvcc.cc:205] Tried to move back new op lower bound from 7317190921390153728 to 7317190921276207104. Current Snapshot: MvccSnapshot[applied={T|T < 7317190921390153728}]
I20260811 04:46:04.811666 5571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:04.811826 5571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:04.814419 5571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Sending DeleteTablet for 3 replicas of tablet 2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:04.815202 5883 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d42ba938b6c43ec89295d3c4a371bc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:04 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:36206
I20260811 04:46:04.815250 6040 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d42ba938b6c43ec89295d3c4a371bc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:04 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:35394
I20260811 04:46:04.815464 5716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d42ba938b6c43ec89295d3c4a371bc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:04 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:54984
I20260811 04:46:04.815824 6157 tablet_replica.cc:330] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: stopping tablet replica
I20260811 04:46:04.815927 6156 tablet_replica.cc:330] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: stopping tablet replica
I20260811 04:46:04.816097 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 5615
I20260811 04:46:04.816172 6157 raft_consensus.cc:2229] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:04.816171 6156 raft_consensus.cc:2229] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:04.816478 6157 raft_consensus.cc:2258] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:04.819846 6156 raft_consensus.cc:2258] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:04.819885 6157 ts_tablet_manager.cc:1905] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:04.821276 6156 ts_tablet_manager.cc:1905] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:04.822378 6157 ts_tablet_manager.cc:1918] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:04.822501 6157 log.cc:1196] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:04.822811 6157 ts_tablet_manager.cc:1939] T 2d42ba938b6c43ec89295d3c4a371bc3 P a96d37f0ad2646a18598ca3d1c0732a1: Deleting consensus metadata
I20260811 04:46:04.824123 6156 ts_tablet_manager.cc:1918] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:04.824142 5552 catalog_manager.cc:4841] TS a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): tablet 2d42ba938b6c43ec89295d3c4a371bc3 (table test-table [id=84ce3cb7bbd943d29cd144c3868fd388]) successfully deleted
I20260811 04:46:04.824225 6156 log.cc:1196] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:04.824563 6156 ts_tablet_manager.cc:1939] T 2d42ba938b6c43ec89295d3c4a371bc3 P 3b39f25aece44c96a43d938c6a99c775: Deleting consensus metadata
I20260811 04:46:04.825857 5553 catalog_manager.cc:4841] TS 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): tablet 2d42ba938b6c43ec89295d3c4a371bc3 (table test-table [id=84ce3cb7bbd943d29cd144c3868fd388]) successfully deleted
W20260811 04:46:04.826259 5551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:45537 (error 108)
I20260811 04:46:04.826579 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 5804
I20260811 04:46:04.828574 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:04.828253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:45537 (local address 127.0.0.1:55006)
0811 04:46:04.828419 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:04.828484 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 04:46:04.828850 5551 catalog_manager.cc:4584] TS f5a6fc5ea5304a39b43c0b1e8c0b3b84 (127.22.144.129:45537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d42ba938b6c43ec89295d3c4a371bc3: Network error: Client connection negotiation failed: client connection to 127.22.144.129:45537: connect: Connection refused (error 111)
I20260811 04:46:04.835991 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 5966
I20260811 04:46:04.844811 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 5528
I20260811 04:46:04.856122 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41843
--webserver_interface=127.22.144.190
--webserver_port=44837
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:04.984681 6160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:04.984973 6160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:04.985019 6160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:04.989428 6160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:46:04.989562 6160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:04.989589 6160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:04.989606 6160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:46:04.989657 6160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:46:04.995188 6160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=44837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:04.996271 6160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:04.997334 6160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:05.006081 6168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:05.006150 6166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.006271 6160 server_base.cc:1034] running on GCE node
W20260811 04:46:05.006067 6165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.006980 6160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:05.008095 6160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:05.009295 6160 hybrid_clock.cc:648] HybridClock initialized: now 1786423565009259 us; error 52 us; skew 500 ppm
Aug 11 04:46:05 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:05.012431 6160 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:46:05.013394 6160 webserver.cc:460] Webserver started at http://127.22.144.190:44837/ using document root <none> and password file <none>
I20260811 04:46:05.014009 6160 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:05.014112 6160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:05.018157 6160 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:05.020376 6176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.021777 6160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:46:05.021927 6160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "733a1b1464144dd29e36fb84388d164b"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "c7e376fe71471d6995f2a04729be3c8b"
server_key_iv: "7e3c467c9d0532474631ddf73ae8000d"
server_key_version: "encryptionkey@0"
I20260811 04:46:05.022496 6160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:05.044100 6160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:05.047458 6160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:05.047688 6160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:05.055367 6160 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:41843
I20260811 04:46:05.055756 6234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:41843 every 8 connection(s)
I20260811 04:46:05.056564 6160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:46:05.061072 6236 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:46:05.062886 6236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap starting.
I20260811 04:46:05.064688 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 6160
I20260811 04:46:05.065212 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:45537
--local_ip_for_outbound_sockets=127.22.144.129
--tserver_master_addrs=127.22.144.190:41843
--webserver_port=44033
--webserver_interface=127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:46:05.082635 6236 log.cc:824] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:05.097913 6236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:46:05.098414 6236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b: Bootstrap complete.
I20260811 04:46:05.104382 6236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:05.104902 6236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 733a1b1464144dd29e36fb84388d164b, State: Initialized, Role: FOLLOWER
I20260811 04:46:05.105465 6236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:05.105660 6236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:46:05.105734 6236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:46:05.105841 6236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 2 FOLLOWER]: Advancing to term 3
I20260811 04:46:05.108476 6236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:05.108850 6236 leader_election.cc:302] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 733a1b1464144dd29e36fb84388d164b; no voters:
I20260811 04:46:05.109266 6236 leader_election.cc:288] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 04:46:05.109412 6241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 3 FOLLOWER]: Leader election won for term 3
I20260811 04:46:05.109730 6241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [term 3 LEADER]: Becoming Leader. State: Replica: 733a1b1464144dd29e36fb84388d164b, State: Running, Role: LEADER
I20260811 04:46:05.110062 6241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } }
I20260811 04:46:05.110611 6236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:46:05.111445 6242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "733a1b1464144dd29e36fb84388d164b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } } }
I20260811 04:46:05.111581 6242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:05.111907 6243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 733a1b1464144dd29e36fb84388d164b. Latest consensus state: current_term: 3 leader_uuid: "733a1b1464144dd29e36fb84388d164b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "733a1b1464144dd29e36fb84388d164b" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41843 } } }
I20260811 04:46:05.111992 6243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 733a1b1464144dd29e36fb84388d164b [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:05.112339 6252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:46:05.117607 6252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:46:05.118355 6252 catalog_manager.cc:1259] Loaded cluster ID: 60cb661fff0d4332879ef0767ff51757
I20260811 04:46:05.118430 6252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 04:46:05.119351 6252 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f71cc6f69ad google::LogMessage::Fail() at ??:0
@ 0x7f71cc6f988a google::LogMessage::SendToLog() at ??:0
@ 0x7f71cc6f64de google::LogMessage::Flush() at ??:0
@ 0x7f71cc6f7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f71d7035fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f71d7036d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f71d7033579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f71d7077640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f71ce9483d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f71ce948c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f71ce94a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f71ce939512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f71cff7d6db start_thread at ??:0
@ 0x7f71cb1f461f clone at ??:0
W20260811 04:46:05.221827 6239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:05.222107 6239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:05.222151 6239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:05.226408 6239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:05.226475 6239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:05.226536 6239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:46:05.232071 6239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:45537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=44033
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:05.233307 6239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:05.234534 6239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:05.244803 6285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.244967 6239 server_base.cc:1034] running on GCE node
W20260811 04:46:05.245071 6286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:05.245247 6289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.245744 6239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:05.246455 6239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:05.247656 6239 hybrid_clock.cc:648] HybridClock initialized: now 1786423565247637 us; error 36 us; skew 500 ppm
Aug 11 04:46:05 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:05.250766 6239 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:46:05.252027 6239 webserver.cc:460] Webserver started at http://127.22.144.129:44033/ using document root <none> and password file <none>
I20260811 04:46:05.252753 6239 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:05.252833 6239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:05.257467 6239 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:05.260283 6297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.261745 6239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 04:46:05.261891 6239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84"
format_stamp: "Formatted at 2026-08-11 04:46:02 on dist-test-slave-ss4n"
server_key: "5e09262d707737001b430383f4a327e4"
server_key_iv: "fc6847f6e5e2ccffeba2a69d415ea6da"
server_key_version: "encryptionkey@0"
I20260811 04:46:05.262491 6239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:05.277340 6239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:05.281075 6239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:05.281311 6239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:05.282017 6239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:05.283270 6305 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 04:46:05.289075 6239 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 04:46:05.289181 6239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 04:46:05.289271 6239 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 04:46:05.291239 6239 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 04:46:05.291321 6239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 04:46:05.291630 6305 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap starting.
I20260811 04:46:05.302767 6239 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:45537
I20260811 04:46:05.302821 6430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:45537 every 8 connection(s)
I20260811 04:46:05.303869 6239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:46:05.305757 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 6239
I20260811 04:46:05.306196 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:38005
--local_ip_for_outbound_sockets=127.22.144.130
--tserver_master_addrs=127.22.144.190:41843
--webserver_port=33223
--webserver_interface=127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:46:05.313302 6305 log.cc:824] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:05.324477 6305 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:46:05.325001 6305 tablet_bootstrap.cc:492] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Bootstrap complete.
I20260811 04:46:05.326524 6305 ts_tablet_manager.cc:1397] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent bootstrapping tablet: real 0.035s user 0.022s sys 0.012s
I20260811 04:46:05.329919 6305 raft_consensus.cc:348] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:05.330490 6305 raft_consensus.cc:729] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a6fc5ea5304a39b43c0b1e8c0b3b84, State: Initialized, Role: FOLLOWER
I20260811 04:46:05.331033 6305 consensus_queue.cc:260] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:05.332228 6305 ts_tablet_manager.cc:1428] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260811 04:46:05.333271 6433 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:46:05.306232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:55597)
0811 04:46:05.307794 (+ 1562us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.307860 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:05.308774 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:05.332489 (+ 23715us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1303,"thread_start_us":151,"threads_started":1}
W20260811 04:46:05.333789 6431 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:41843 (0 consecutive failures): Network error: Failed to ping master at 127.22.144.190:41843: Client connection negotiation failed: client connection to 127.22.144.190:41843: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 04:46:05.334537 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.334301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:34813)
0811 04:46:05.334422 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.334482 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 04:46:05.335244 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.335046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:49385)
0811 04:46:05.335151 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.335195 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 04:46:05.335491 6431 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 04:46:05.435793 6434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:05.436084 6434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:05.436128 6434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:05.440384 6434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:05.440459 6434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:05.440521 6434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:46:05.445876 6434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:38005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=33223
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:05.447037 6434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:05.448127 6434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:05.458261 6445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.458328 6434 server_base.cc:1034] running on GCE node
W20260811 04:46:05.458276 6444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:05.458271 6447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.459116 6434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:05.459720 6434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:05.460886 6434 hybrid_clock.cc:648] HybridClock initialized: now 1786423565460853 us; error 46 us; skew 500 ppm
Aug 11 04:46:05 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:05.463771 6434 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:46:05.464697 6434 webserver.cc:460] Webserver started at http://127.22.144.130:33223/ using document root <none> and password file <none>
I20260811 04:46:05.465245 6434 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:05.465305 6434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:05.469828 6434 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 04:46:05.472056 6457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.473398 6434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:05.473603 6434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "3b39f25aece44c96a43d938c6a99c775"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "6f1d5582571aa6e6f1c7739344cb0aa8"
server_key_iv: "d3ef70265fbec15f1e4dce76afe513ff"
server_key_version: "encryptionkey@0"
I20260811 04:46:05.474170 6434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:05.488723 6434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:05.492328 6434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:05.492540 6434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:05.493185 6434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:05.494103 6434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:05.494150 6434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.494212 6434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:05.494244 6434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.504846 6434 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:38005
I20260811 04:46:05.504878 6588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:38005 every 8 connection(s)
I20260811 04:46:05.506027 6434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:46:05.509684 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 6434
I20260811 04:46:05.510200 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:39221
--local_ip_for_outbound_sockets=127.22.144.131
--tserver_master_addrs=127.22.144.190:41843
--webserver_port=44871
--webserver_interface=127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:41415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:46:05.510313 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.508260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:34329)
0811 04:46:05.508734 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.508858 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
W20260811 04:46:05.510795 6589 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:41843 (0 consecutive failures): Network error: Failed to ping master at 127.22.144.190:41843: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
I20260811 04:46:05.511487 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.511257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:35647)
0811 04:46:05.511381 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.511430 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 04:46:05.512297 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.512013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:38603)
0811 04:46:05.512159 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.512223 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 04:46:05.512530 6589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 04:46:05.627492 6592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:05.627772 6592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:05.627818 6592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:05.632028 6592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:05.632108 6592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:05.632198 6592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:46:05.637749 6592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:41415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:39221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=44871
--tserver_master_addrs=127.22.144.190:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:05.638890 6592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:05.639976 6592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:05.650416 6603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:05.650642 6598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.650451 6592 server_base.cc:1034] running on GCE node
W20260811 04:46:05.650470 6606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:05.651254 6592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:05.651929 6592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:05.653132 6592 hybrid_clock.cc:648] HybridClock initialized: now 1786423565653095 us; error 51 us; skew 500 ppm
Aug 11 04:46:05 dist-test-slave-ss4n krb5kdc[4860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:05.656112 6592 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:46:05.657021 6592 webserver.cc:460] Webserver started at http://127.22.144.131:44871/ using document root <none> and password file <none>
I20260811 04:46:05.657717 6592 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:05.657821 6592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:05.661965 6592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 04:46:05.664516 6614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.666141 6592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:05.666299 6592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "a96d37f0ad2646a18598ca3d1c0732a1"
format_stamp: "Formatted at 2026-08-11 04:46:03 on dist-test-slave-ss4n"
server_key: "867fb1553d0194710a2abe739cf5df64"
server_key_iv: "be3fc8ca69391b83789fbd0e8268fbd8"
server_key_version: "encryptionkey@0"
I20260811 04:46:05.666882 6592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:05.680763 6592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:05.684423 6592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:05.684664 6592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:05.685392 6592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:05.686333 6592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:05.686406 6592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.686481 6592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:05.686534 6592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:05.696640 6592 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:39221
I20260811 04:46:05.696674 6738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:39221 every 8 connection(s)
I20260811 04:46:05.697889 6592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:46:05.698546 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 6592
I20260811 04:46:05.702759 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.700660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:37879)
0811 04:46:05.701258 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.701378 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":156,"threads_started":1}
W20260811 04:46:05.703344 6739 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:41843 (0 consecutive failures): Network error: Failed to ping master at 127.22.144.190:41843: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
I20260811 04:46:05.704018 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.703788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:54675)
0811 04:46:05.703906 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.703955 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 04:46:05.704736 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:05.704526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:51353)
0811 04:46:05.704641 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:05.704684 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 04:46:05.704964 6739 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 04:46:06.336750 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.336208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:38871)
0811 04:46:06.336574 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:06.336638 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":81,"threads_started":1}
I20260811 04:46:06.513716 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.513137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:47597)
0811 04:46:06.513428 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:06.513603 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":89,"threads_started":1}
I20260811 04:46:06.574280 6745 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:06.574536 6745 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:06.575781 6745 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
Aug 11 04:46:06 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.130@KRBTEST.COM
Aug 11 04:46:06 dist-test-slave-ss4n krb5kdc[4860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423565, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.131@KRBTEST.COM
I20260811 04:46:06.597033 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.576065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38005 (local address 127.22.144.129:34947)
0811 04:46:06.576578 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:06.576592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:06.576689 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:06.578485 (+ 1796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:06.578491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:06.579126 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:06.579557 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:06.579573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.580860 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.580864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:06.581802 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:06.581813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.582571 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.582575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:06.582792 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:06.588976 (+ 6184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:06.589025 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:06.591143 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:06.594092 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:06.594100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:06.594115 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:06.594575 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:06.595023 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:06.595026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:06.595027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:06.595145 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:06.595859 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:06.595866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:06.596169 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 04:46:06.596372 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 04:46:06.596797 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":348,"spinlock_wait_cycles":23040,"thread_start_us":69,"threads_started":1}
I20260811 04:46:06.597051 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.576270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:53893 (local address 127.22.144.131:39221)
0811 04:46:06.576720 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:06.576731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:06.576768 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:46:06.577668 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:06.577684 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:06.578222 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:06.578544 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:06.579743 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.580787 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.581944 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.582469 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.582661 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:06.591850 (+ 9189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:06.591856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:06.591869 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:06.591904 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:06.594112 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:06.594706 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:06.594714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:06.594729 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:06.594858 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:06.595287 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:06.595289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:06.595291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:06.595723 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:06.595883 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:06.596537 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 04:46:06.596834 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":127,"threads_started":1}
I20260811 04:46:06.597119 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.576065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39221 (local address 127.22.144.129:53893)
0811 04:46:06.576221 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:06.576239 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:06.576369 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:06.578487 (+ 2118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:06.578491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:06.579200 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:06.579562 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:06.579573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.580947 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.580953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:06.581806 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:06.581817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.582612 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.582616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:06.582793 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:06.588979 (+ 6186us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:06.589005 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:06.591575 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:06.594253 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:06.594256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:06.594258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:06.594563 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:06.594972 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:06.594978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:06.594984 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:06.595157 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:06.595861 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:06.595871 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:06.596172 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 04:46:06.596371 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 04:46:06.596907 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":290,"spinlock_wait_cycles":36224}
I20260811 04:46:06.597084 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.576270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:34947 (local address 127.22.144.130:38005)
0811 04:46:06.576673 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:06.576709 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:06.576766 (+ 57us) server_negotiation.cc:407] Connection header received
0811 04:46:06.577556 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:06.577568 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:06.578097 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:06.578431 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:06.579741 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.580715 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.581933 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:06.582438 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:06.582638 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:06.591486 (+ 8848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:06.591496 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:06.591518 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:06.591633 (+ 115us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:06.593938 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:06.594735 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:06.594741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:06.594745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:06.594867 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:06.595256 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:06.595259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:06.595262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:06.595724 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:06.595885 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:06.596531 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 04:46:06.596865 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":81,"threads_started":1}
W20260811 04:46:06.600090 6301 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
W20260811 04:46:06.600371 6302 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:06.600459 6302 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters: 3b39f25aece44c96a43d938c6a99c775, a96d37f0ad2646a18598ca3d1c0732a1
I20260811 04:46:06.600767 6745 raft_consensus.cc:2726] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:46:06.706115 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:06.705612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:49713)
0811 04:46:06.705917 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:06.705981 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":96,"threads_started":1}
I20260811 04:46:07.338089 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:07.337657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:45485)
0811 04:46:07.337927 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:07.337988 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260811 04:46:07.514971 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:07.514516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:36979)
0811 04:46:07.514804 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:07.514874 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":79,"threads_started":1}
I20260811 04:46:07.707568 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:07.707029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:44623)
0811 04:46:07.707362 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:07.707440 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":78,"threads_started":1}
I20260811 04:46:08.204068 6757 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:08.204236 6757 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:08.204752 6757 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
W20260811 04:46:08.205710 6301 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
W20260811 04:46:08.205840 6302 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:08.205945 6302 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters: 3b39f25aece44c96a43d938c6a99c775, a96d37f0ad2646a18598ca3d1c0732a1
I20260811 04:46:08.206149 6757 raft_consensus.cc:2726] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:46:08.339462 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:08.338989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:35433)
0811 04:46:08.339298 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:08.339362 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":83,"threads_started":1}
I20260811 04:46:08.516366 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:08.515832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:58379)
0811 04:46:08.516128 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:08.516207 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260811 04:46:08.708801 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:08.708380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:42245)
0811 04:46:08.708636 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:08.708697 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260811 04:46:09.340822 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:09.340344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:41711)
0811 04:46:09.340651 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:09.340712 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":98,"threads_started":1}
I20260811 04:46:09.517930 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:09.517301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:49119)
0811 04:46:09.517704 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:09.517769 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":103,"threads_started":1}
I20260811 04:46:09.710186 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:09.709705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:44685)
0811 04:46:09.709997 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:09.710080 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260811 04:46:10.342470 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:10.341946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:59813)
0811 04:46:10.342299 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:10.342366 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
I20260811 04:46:10.519379 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:10.518901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:37091)
0811 04:46:10.519190 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:10.519257 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":90,"threads_started":1}
I20260811 04:46:10.711607 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:10.711114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:59211)
0811 04:46:10.711405 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:10.711480 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
I20260811 04:46:11.289778 6767 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:11.289917 6767 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:11.290524 6767 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
W20260811 04:46:11.291361 6302 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
W20260811 04:46:11.291481 6301 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:11.291611 6301 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters: 3b39f25aece44c96a43d938c6a99c775, a96d37f0ad2646a18598ca3d1c0732a1
I20260811 04:46:11.291781 6767 raft_consensus.cc:2726] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:46:11.343937 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:11.343418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:35849)
0811 04:46:11.343771 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:11.343836 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":163,"threads_started":1}
I20260811 04:46:11.520951 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:11.520423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:54027)
0811 04:46:11.520754 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:11.520821 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":96,"threads_started":1}
I20260811 04:46:11.712913 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:11.712443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:33287)
0811 04:46:11.712726 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:11.712808 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260811 04:46:12.345333 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:12.344841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:52935)
0811 04:46:12.345164 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:12.345230 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":83,"threads_started":1}
I20260811 04:46:12.522315 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:12.521787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:46761)
0811 04:46:12.522140 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:12.522214 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":93,"threads_started":1}
I20260811 04:46:12.714447 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:12.713858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:58455)
0811 04:46:12.714276 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:12.714343 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":121,"threads_started":1}
I20260811 04:46:13.346822 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:13.346320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:49125)
0811 04:46:13.346645 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:13.346715 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":89,"threads_started":1}
I20260811 04:46:13.523880 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:13.523401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:53367)
0811 04:46:13.523723 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:13.523780 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":82,"threads_started":1}
I20260811 04:46:13.715817 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:13.715381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:52593)
0811 04:46:13.715662 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:13.715716 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260811 04:46:14.348376 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:14.347915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:40363)
0811 04:46:14.348210 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:14.348274 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":105,"threads_started":1}
I20260811 04:46:14.525410 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:14.524889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:52829)
0811 04:46:14.525242 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:14.525310 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":88,"threads_started":1}
I20260811 04:46:14.717131 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:14.716696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:41907)
0811 04:46:14.716959 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:14.717016 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260811 04:46:15.350160 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:15.349601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:56027)
0811 04:46:15.349918 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:15.349997 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":123,"threads_started":1}
I20260811 04:46:15.527100 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:15.526606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:53775)
0811 04:46:15.526896 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:15.526966 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":119,"threads_started":1}
I20260811 04:46:15.718508 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:15.718068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:34969)
0811 04:46:15.718335 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:15.718398 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260811 04:46:16.309098 6787 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:16.309238 6787 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:16.309708 6787 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
W20260811 04:46:16.310621 6301 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
W20260811 04:46:16.310725 6302 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:16.310850 6302 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters: 3b39f25aece44c96a43d938c6a99c775, a96d37f0ad2646a18598ca3d1c0732a1
I20260811 04:46:16.311021 6787 raft_consensus.cc:2726] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:46:16.351558 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:16.351069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:60563)
0811 04:46:16.351382 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:16.351446 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
I20260811 04:46:16.528509 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:16.528021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:52245)
0811 04:46:16.528323 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:16.528386 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":88,"threads_started":1}
I20260811 04:46:16.720050 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:16.719499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:51661)
0811 04:46:16.719849 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:16.719919 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
I20260811 04:46:17.353022 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:17.352492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:40283)
0811 04:46:17.352840 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:17.352914 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260811 04:46:17.529927 6792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:17.529404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:42003)
0811 04:46:17.529751 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:17.529817 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":137,"threads_started":1}
I20260811 04:46:17.721401 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:17.720975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:50963)
0811 04:46:17.721238 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:17.721297 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
I20260811 04:46:18.354588 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:18.354070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:46025)
0811 04:46:18.354373 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:18.354436 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
I20260811 04:46:18.531788 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:18.531085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:44015)
0811 04:46:18.531570 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:18.531676 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":251,"threads_started":1}
I20260811 04:46:18.722828 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:18.722368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:43371)
0811 04:46:18.722650 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:18.722726 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
I20260811 04:46:19.355970 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:19.355470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:41855)
0811 04:46:19.355802 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:19.355868 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":85,"threads_started":1}
I20260811 04:46:19.533252 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:19.532779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:41575)
0811 04:46:19.533060 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:19.533152 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260811 04:46:19.724279 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:19.723777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.131:50593)
0811 04:46:19.724078 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:19.724157 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":84,"threads_started":1}
I20260811 04:46:20.357803 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:20.357259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.129:44789)
0811 04:46:20.357635 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:20.357703 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":88,"threads_started":1}
I20260811 04:46:20.465768 6804 raft_consensus.cc:482] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:20.465960 6804 raft_consensus.cc:504] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b39f25aece44c96a43d938c6a99c775" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38005 } } peers { permanent_uuid: "a96d37f0ad2646a18598ca3d1c0732a1" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39221 } } peers { permanent_uuid: "f5a6fc5ea5304a39b43c0b1e8c0b3b84" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 45537 } }
I20260811 04:46:20.466506 6804 leader_election.cc:288] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005), a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221)
W20260811 04:46:20.467383 6302 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b39f25aece44c96a43d938c6a99c775 (127.22.144.130:38005): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
W20260811 04:46:20.467510 6301 leader_election.cc:341] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a96d37f0ad2646a18598ca3d1c0732a1 (127.22.144.131:39221): Not found: Tablet not found: 2d42ba938b6c43ec89295d3c4a371bc3
I20260811 04:46:20.467609 6301 leader_election.cc:302] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5a6fc5ea5304a39b43c0b1e8c0b3b84; no voters: 3b39f25aece44c96a43d938c6a99c775, a96d37f0ad2646a18598ca3d1c0732a1
I20260811 04:46:20.467761 6804 raft_consensus.cc:2726] T 2d42ba938b6c43ec89295d3c4a371bc3 P f5a6fc5ea5304a39b43c0b1e8c0b3b84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:46:20.534912 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:20.534439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41843 (local address 127.22.144.130:60583)
0811 04:46:20.534729 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:20.534796 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":74,"threads_started":1}
I20260811 04:46:20.700968 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 6239
I20260811 04:46:20.711074 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 6434
I20260811 04:46:20.719954 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 6592
2026-08-11T04:46:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19553 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:46:20 dist-test-slave-ss4n krb5kdc[6809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:46:20 dist-test-slave-ss4n krb5kdc[6809](info): set up 2 sockets
Aug 11 04:46:20 dist-test-slave-ss4n krb5kdc[6809](info): commencing operation
krb5kdc: starting...
W20260811 04:46:21.868860 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:46:21 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:46:22.009197 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39221
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:39221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:22.123287 6827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:22.123631 6827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:22.123718 6827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:22.127866 6827 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 04:46:22.127921 6827 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 04:46:22.127938 6827 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 04:46:22.127959 6827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:46:22.128019 6827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:22.128052 6827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:22.128093 6827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:46:22.128129 6827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:46:22.133647 6827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:39221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:22.134763 6827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:22.135825 6827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:22.145725 6833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.145799 6827 server_base.cc:1034] running on GCE node
W20260811 04:46:22.145813 6832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:22.145727 6835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.146768 6827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:22.148114 6827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:22.149309 6827 hybrid_clock.cc:648] HybridClock initialized: now 1786423582149289 us; error 34 us; skew 500 ppm
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:22.152364 6827 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:46:22.153383 6827 webserver.cc:460] Webserver started at http://127.22.144.190:37467/ using document root <none> and password file <none>
I20260811 04:46:22.153985 6827 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:22.154049 6827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:22.154278 6827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:22.156338 6827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "aa3023cb05d249179715be8e33d100a6"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "44e55db81da2eda31eb097ec37c2899f"
server_key_iv: "a4fe146cfe2f2e419ebc24640f9f8c6b"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.156944 6827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "aa3023cb05d249179715be8e33d100a6"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "44e55db81da2eda31eb097ec37c2899f"
server_key_iv: "a4fe146cfe2f2e419ebc24640f9f8c6b"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.160712 6827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 04:46:22.163331 6842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.164590 6827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 04:46:22.164714 6827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "aa3023cb05d249179715be8e33d100a6"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "44e55db81da2eda31eb097ec37c2899f"
server_key_iv: "a4fe146cfe2f2e419ebc24640f9f8c6b"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.164815 6827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:22.184881 6827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:22.188395 6827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:22.188606 6827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:22.196058 6827 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:39221
I20260811 04:46:22.196383 6901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:39221 every 8 connection(s)
I20260811 04:46:22.197062 6827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:46:22.199954 6902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:22.205569 6902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap starting.
I20260811 04:46:22.206372 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 6827
I20260811 04:46:22.206525 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:46:22.206799 23106 external_mini_cluster.cc:1428] Setting key 6ecf77923788c789349abdc61de8a3b5
I20260811 04:46:22.208381 6902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:22.209159 6902 log.cc:824] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:22.211462 6902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: No bootstrap required, opened a new log
I20260811 04:46:22.215703 6902 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:22.215957 6902 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:22.216061 6902 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3023cb05d249179715be8e33d100a6, State: Initialized, Role: FOLLOWER
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:22.216630 6902 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:22.216820 6902 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:46:22.216897 6902 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:46:22.217012 6902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:22.218268 6902 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:22.218628 6902 leader_election.cc:302] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3023cb05d249179715be8e33d100a6; no voters:
I20260811 04:46:22.218968 6902 leader_election.cc:288] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:46:22.219094 6908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:22.219321 6908 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 LEADER]: Becoming Leader. State: Replica: aa3023cb05d249179715be8e33d100a6, State: Running, Role: LEADER
I20260811 04:46:22.219682 6908 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:22.220331 6902 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:46:22.221370 6909 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa3023cb05d249179715be8e33d100a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } } }
I20260811 04:46:22.221519 6909 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:22.221930 6910 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3023cb05d249179715be8e33d100a6. Latest consensus state: current_term: 1 leader_uuid: "aa3023cb05d249179715be8e33d100a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } } }
I20260811 04:46:22.222031 6910 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:22.222397 6918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:46:22.223120 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:22.207969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.22.144.190:39221)
0811 04:46:22.208487 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:22.208516 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:22.208567 (+ 51us) server_negotiation.cc:407] Connection header received
0811 04:46:22.209199 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:22.209217 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:22.209698 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:22.210029 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:22.210743 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.212192 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.213061 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.213816 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.213994 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:22.216894 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:22.216904 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:22.216939 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:22.216978 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:22.218920 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:22.219548 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:22.219554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:22.219559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:22.219699 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:22.220026 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:22.220030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:22.220032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:22.220577 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:22.220786 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:22.221137 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 04:46:22.221531 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":174,"threads_started":1}
I20260811 04:46:22.225260 6918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:46:22.230878 6918 catalog_manager.cc:1347] Generated new cluster ID: b6de7b353ed54660ab299ff74c243784
I20260811 04:46:22.230975 6918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:46:22.247942 6918 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:46:22.249045 6918 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:46:22.255518 6918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Generated new TSK 0
I20260811 04:46:22.256244 6918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:46:22.297477 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39221
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:22.456212 6933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:22.456488 6933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:22.456537 6933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:22.460534 6933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:22.460594 6933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:22.460656 6933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:46:22.466215 6933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:22.467258 6933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:22.468235 6933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:22.478464 6944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:22.478489 6939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.478710 6933 server_base.cc:1034] running on GCE node
W20260811 04:46:22.478464 6940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.479225 6933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:22.479800 6933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:22.480969 6933 hybrid_clock.cc:648] HybridClock initialized: now 1786423582480937 us; error 45 us; skew 500 ppm
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:22.483932 6933 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:46:22.484875 6933 webserver.cc:460] Webserver started at http://127.22.144.129:32841/ using document root <none> and password file <none>
I20260811 04:46:22.485466 6933 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:22.485617 6933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:22.485846 6933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:22.487958 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "f28491784a714619a4f2274a78752937"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "049db109d7222eeade027beece84116a"
server_key_iv: "e97fa7dc9fcb25edc4489c95becaece4"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.488543 6933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "f28491784a714619a4f2274a78752937"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "049db109d7222eeade027beece84116a"
server_key_iv: "e97fa7dc9fcb25edc4489c95becaece4"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.492520 6933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 04:46:22.495184 6952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.496763 6933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:46:22.496917 6933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "f28491784a714619a4f2274a78752937"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "049db109d7222eeade027beece84116a"
server_key_iv: "e97fa7dc9fcb25edc4489c95becaece4"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.497056 6933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:22.512425 6933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:22.515929 6933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:22.516172 6933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:22.516804 6933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:22.517755 6933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:22.517827 6933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.517900 6933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:22.517943 6933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.529163 6933 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:43785
I20260811 04:46:22.529208 7083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:43785 every 8 connection(s)
I20260811 04:46:22.530154 6933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:46:22.537598 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 6933
I20260811 04:46:22.537744 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:46:22.538085 23106 external_mini_cluster.cc:1428] Setting key 2eb79b23fd0804c0f42851c4e4ae3b40
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:22.546300 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:22.532199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:52181 (local address 127.22.144.190:39221)
0811 04:46:22.532336 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:22.532339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:22.533140 (+ 801us) server_negotiation.cc:407] Connection header received
0811 04:46:22.534692 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:22.534696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:22.534757 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:22.534838 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:22.536030 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.536582 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.538087 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.538411 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.538508 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:22.541832 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:22.541839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:22.541845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:22.541882 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:22.543457 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:22.544164 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:22.544169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:22.544171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:22.544284 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:22.544801 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:22.544805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:22.544807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:22.545066 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:22.545218 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:22.545927 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 04:46:22.546087 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 04:46:22.547197 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:22.532504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:52181)
0811 04:46:22.532982 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:22.533029 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:22.534460 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:22.535008 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:22.535014 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:22.535376 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:22.535862 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:22.535876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.536720 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.536729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:22.537930 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:22.537943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.538761 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.538767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:22.539046 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:22.539846 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:22.539871 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:22.541658 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:22.543601 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:22.543607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:22.543621 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:22.544006 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:22.544491 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:22.544494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:22.544495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:22.544664 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:22.545232 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:22.545243 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:22.545613 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 04:46:22.545835 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 04:46:22.546252 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260811 04:46:22.548755 7084 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39221
I20260811 04:46:22.549080 7084 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:46:22.549750 7084 heartbeater.cc:511] Master 127.22.144.190:39221 requested a full tablet report, sending...
I20260811 04:46:22.551429 6862 ts_manager.cc:194] Registered new tserver with Master: f28491784a714619a4f2274a78752937 (127.22.144.129:43785)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:46:22.552791 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:52181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:46:22.572149 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39221
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:22.688722 7089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:22.689002 7089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:22.689054 7089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:22.693233 7089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:22.693303 7089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:22.693364 7089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:46:22.698864 7089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:22.699946 7089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:22.701036 7089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:22.711836 7094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.711943 7089 server_base.cc:1034] running on GCE node
W20260811 04:46:22.711884 7098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:22.711884 7095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.712594 7089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:22.713286 7089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:22.714545 7089 hybrid_clock.cc:648] HybridClock initialized: now 1786423582714520 us; error 45 us; skew 500 ppm
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:22.718195 7089 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:46:22.719266 7089 webserver.cc:460] Webserver started at http://127.22.144.130:34797/ using document root <none> and password file <none>
I20260811 04:46:22.719899 7089 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:22.719978 7089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:22.720219 7089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:22.722857 7089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "fc980b2b626e7580b93136400bc65b48"
server_key_iv: "6d1440fc35308f33f37843c5c0a12a9a"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.723595 7089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "fc980b2b626e7580b93136400bc65b48"
server_key_iv: "6d1440fc35308f33f37843c5c0a12a9a"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.727756 7089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:46:22.730350 7106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.731607 7089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:46:22.731731 7089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "fc980b2b626e7580b93136400bc65b48"
server_key_iv: "6d1440fc35308f33f37843c5c0a12a9a"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.731843 7089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:22.747917 7089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:22.751405 7089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:22.751618 7089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:22.752245 7089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:22.753088 7089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:22.753144 7089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.753185 7089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:22.753201 7089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.763201 7089 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:34145
I20260811 04:46:22.763221 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:34145 every 8 connection(s)
I20260811 04:46:22.764118 7089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:46:22.770023 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 7089
I20260811 04:46:22.770185 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:46:22.770479 23106 external_mini_cluster.cc:1428] Setting key d6b2210148445faa931b1c6a21ec7162
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:22.779644 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:22.766012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:51423 (local address 127.22.144.190:39221)
0811 04:46:22.766160 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:22.766164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:22.767481 (+ 1317us) server_negotiation.cc:407] Connection header received
0811 04:46:22.768396 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:22.768400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:22.768460 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:22.768542 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:22.769782 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.770435 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.771797 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.772023 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.772147 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:22.775006 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:22.775012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:22.775017 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:22.775052 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:22.776831 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:22.777717 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:22.777720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:22.777721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:22.777804 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:22.778266 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:22.778269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:22.778270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:22.778456 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:22.778531 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:22.779391 (+ 860us) server_negotiation.cc:299] Negotiation successful
0811 04:46:22.779504 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 04:46:22.781085 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:22.766422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:51423)
0811 04:46:22.767328 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:22.767373 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:22.768184 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:22.768717 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:22.768723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:22.769080 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:22.769638 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:22.769654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.770564 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.770567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:22.771621 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:22.771631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:22.772199 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:22.772203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:22.772374 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:22.772994 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:22.773010 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:22.774810 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:22.776997 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:22.777008 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:22.777031 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:22.777593 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:22.777914 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:22.777920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:22.777924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:22.778154 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:22.778582 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:22.778592 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:22.779047 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 04:46:22.779318 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 04:46:22.779813 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":553,"threads_started":1}
I20260811 04:46:22.782675 7236 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39221
I20260811 04:46:22.782956 7236 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:46:22.783525 7236 heartbeater.cc:511] Master 127.22.144.190:39221 requested a full tablet report, sending...
I20260811 04:46:22.784929 6862 ts_manager.cc:194] Registered new tserver with Master: 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145)
I20260811 04:46:22.785719 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:51423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:46:22.805063 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39221
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:22.925256 7242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:22.925580 7242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:22.925665 7242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:22.929675 7242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:22.929768 7242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:22.929857 7242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:46:22.935369 7242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:22.936548 7242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:22.937649 7242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:22.948096 7252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:22.948091 7251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.948143 7242 server_base.cc:1034] running on GCE node
W20260811 04:46:22.948091 7255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:22.948854 7242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:22.949608 7242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:22.950781 7242 hybrid_clock.cc:648] HybridClock initialized: now 1786423582950748 us; error 46 us; skew 500 ppm
Aug 11 04:46:22 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:22.953994 7242 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:46:22.954949 7242 webserver.cc:460] Webserver started at http://127.22.144.131:34497/ using document root <none> and password file <none>
I20260811 04:46:22.955543 7242 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:22.955608 7242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:22.955848 7242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:46:22.957983 7242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "db1988ee561a4676a0f347629caebb92"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "9c84b9318d865142df002da5fbb15b8b"
server_key_iv: "d5b15633105df615977e41f2ea3cafae"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.958498 7242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "db1988ee561a4676a0f347629caebb92"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "9c84b9318d865142df002da5fbb15b8b"
server_key_iv: "d5b15633105df615977e41f2ea3cafae"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.962499 7242 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:46:22.964875 7263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.966244 7242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 04:46:22.966394 7242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "db1988ee561a4676a0f347629caebb92"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "9c84b9318d865142df002da5fbb15b8b"
server_key_iv: "d5b15633105df615977e41f2ea3cafae"
server_key_version: "encryptionkey@0"
I20260811 04:46:22.966517 7242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:22.985719 7242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:22.988785 7242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:22.989092 7242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:22.989838 7242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:22.990658 7242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:22.990706 7242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:22.990746 7242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:22.990770 7242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:23.000068 7242 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:44319
I20260811 04:46:23.000092 7401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:44319 every 8 connection(s)
I20260811 04:46:23.001122 7242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:46:23.003139 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 7242
I20260811 04:46:23.003263 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:46:23.003602 23106 external_mini_cluster.cc:1428] Setting key b6ae931ba7ac7b68f52a078fd19b71a1
Aug 11 04:46:23 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:23.015988 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.003038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:38621 (local address 127.22.144.190:39221)
0811 04:46:23.003188 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.003191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.004322 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 04:46:23.006131 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.006134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.006190 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.006273 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:23.007486 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.008021 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.009256 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.009533 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.009658 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.012301 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:23.012310 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:23.012313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:23.012349 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:23.013779 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:23.014312 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:23.014315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:23.014317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:23.014404 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:23.014760 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:23.014765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:23.014767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:23.015031 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:23.015145 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.015671 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.015810 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 04:46:23.017021 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.003586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:38621)
0811 04:46:23.004177 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.004224 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:23.005891 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:23.006457 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:23.006464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:23.006841 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:23.007340 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.007354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.008160 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.008163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.009112 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.009122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.009696 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.009700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.009893 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.010534 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:23.010552 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:23.012106 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:23.013898 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:23.013904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:23.013921 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:23.014198 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:23.014498 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:23.014502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:23.014504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:23.014659 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:23.015149 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:23.015158 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:23.015420 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 04:46:23.015618 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 04:46:23.015954 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":146,"threads_started":1}
I20260811 04:46:23.018339 7403 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39221
I20260811 04:46:23.018546 7403 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:23.018997 7403 heartbeater.cc:511] Master 127.22.144.190:39221 requested a full tablet report, sending...
I20260811 04:46:23.019970 6862 ts_manager.cc:194] Registered new tserver with Master: db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
I20260811 04:46:23.020593 6862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:38621
I20260811 04:46:23.029987 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:46:23.039600 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.032039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.22.144.190:39221)
0811 04:46:23.032187 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.032191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.032359 (+ 168us) server_negotiation.cc:407] Connection header received
0811 04:46:23.032446 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.032448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.032491 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.032555 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:23.033244 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.033830 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.034610 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.034819 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.034900 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.036310 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:23.036313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:23.036315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:23.036340 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:23.037939 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:23.038327 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:23.038330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:23.038331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:23.038410 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:23.038639 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:23.038641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:23.038642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:23.038867 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:23.038969 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.039186 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.039375 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:46:23.042662 6862 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:46:23.045749 6862 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:46:23.060436 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.054163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34145 (local address 127.0.0.1:46706)
0811 04:46:23.054965 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.055015 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:23.055188 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:23.055911 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:23.055919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:23.055950 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:23.056086 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.056096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.057619 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.057623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.058766 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.058777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.059877 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.059881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.059989 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.060029 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 04:46:23.060156 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 04:46:23.060276 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":53,"thread_start_us":81,"threads_started":1}
I20260811 04:46:23.060472 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.054298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.22.144.130:34145)
0811 04:46:23.054702 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.054714 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.055102 (+ 388us) server_negotiation.cc:407] Connection header received
0811 04:46:23.055502 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.055509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.055730 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.055906 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:23.056197 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.057436 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.058892 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.059752 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.059883 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.060018 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.060189 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.060301 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":107,"threads_started":1}
I20260811 04:46:23.061153 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.054955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41562 (local address 127.22.144.131:44319)
0811 04:46:23.055592 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.055602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.055620 (+ 18us) server_negotiation.cc:407] Connection header received
0811 04:46:23.055685 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.055688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.055849 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.055997 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:23.056379 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.057240 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.058950 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.059752 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.059883 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.060018 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.060861 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.060979 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":117,"threads_started":1}
I20260811 04:46:23.061455 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.054274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44319 (local address 127.0.0.1:41562)
0811 04:46:23.054983 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.055026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:23.055160 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:23.056145 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:23.056149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:23.056171 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:23.056268 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.056273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.057369 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.057374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.058808 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.058817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.060109 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.060134 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.060227 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.060251 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:46:23.060322 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 04:46:23.060393 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":116,"thread_start_us":107,"threads_started":1}
I20260811 04:46:23.061839 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.055342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.22.144.129:43785)
0811 04:46:23.055717 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.055723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.055953 (+ 230us) server_negotiation.cc:407] Connection header received
0811 04:46:23.056155 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.056162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.056341 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.056485 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:23.056727 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.057983 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.059696 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.060489 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.061023 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.061149 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.061565 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.061688 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":97,"threads_started":1}
I20260811 04:46:23.062101 7414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.054556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43785 (local address 127.0.0.1:40210)
0811 04:46:23.055866 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.055879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:23.056022 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:23.056473 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:23.056476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:23.056496 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:23.056604 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.056611 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.058803 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.058807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.059563 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.059572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.060900 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.060904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.061289 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.061312 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:46:23.061382 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 04:46:23.061447 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"thread_start_us":98,"threads_started":1}
I20260811 04:46:23.064028 7164 tablet_service.cc:1467] Processing CreateTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 (DEFAULT_TABLE table=test-table [id=33bbe3e1040747058dd4af81cd7ae5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:23.064380 7010 tablet_service.cc:1467] Processing CreateTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 (DEFAULT_TABLE table=test-table [id=33bbe3e1040747058dd4af81cd7ae5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:23.064740 7322 tablet_service.cc:1467] Processing CreateTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 (DEFAULT_TABLE table=test-table [id=33bbe3e1040747058dd4af81cd7ae5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:23.065243 7164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b913f09fd9141e09b97c8e55be5d5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:23.065400 7010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b913f09fd9141e09b97c8e55be5d5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:23.065877 7322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b913f09fd9141e09b97c8e55be5d5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:23.072094 7423 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Bootstrap starting.
I20260811 04:46:23.073330 7425 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Bootstrap starting.
I20260811 04:46:23.073789 7424 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Bootstrap starting.
I20260811 04:46:23.074843 7423 tablet_bootstrap.cc:654] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:23.075387 7425 tablet_bootstrap.cc:654] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:23.075740 7423 log.cc:824] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:23.076243 7424 tablet_bootstrap.cc:654] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:23.076632 7425 log.cc:824] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:23.077042 7424 log.cc:824] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:23.078001 7423 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: No bootstrap required, opened a new log
I20260811 04:46:23.078193 7423 ts_tablet_manager.cc:1397] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 04:46:23.078538 7425 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: No bootstrap required, opened a new log
I20260811 04:46:23.078734 7425 ts_tablet_manager.cc:1397] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:46:23.078955 7424 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: No bootstrap required, opened a new log
I20260811 04:46:23.079135 7424 ts_tablet_manager.cc:1397] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 04:46:23.082026 7425 raft_consensus.cc:348] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.082326 7425 raft_consensus.cc:374] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:23.082392 7425 raft_consensus.cc:729] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1988ee561a4676a0f347629caebb92, State: Initialized, Role: FOLLOWER
I20260811 04:46:23.082852 7425 consensus_queue.cc:260] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.082876 7423 raft_consensus.cc:348] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.082940 7424 raft_consensus.cc:348] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.083148 7423 raft_consensus.cc:374] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:23.083163 7424 raft_consensus.cc:374] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:23.083217 7424 raft_consensus.cc:729] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae5ee8ba6b34f24922a920cfc6c3fa5, State: Initialized, Role: FOLLOWER
I20260811 04:46:23.083217 7423 raft_consensus.cc:729] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28491784a714619a4f2274a78752937, State: Initialized, Role: FOLLOWER
I20260811 04:46:23.083696 7403 heartbeater.cc:503] Master 127.22.144.190:39221 was elected leader, sending a full tablet report...
I20260811 04:46:23.083642 7424 consensus_queue.cc:260] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.083642 7423 consensus_queue.cc:260] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.083900 7425 ts_tablet_manager.cc:1428] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 04:46:23.084556 7424 ts_tablet_manager.cc:1428] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 04:46:23.084556 7423 ts_tablet_manager.cc:1428] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:46:23.084894 7084 heartbeater.cc:503] Master 127.22.144.190:39221 was elected leader, sending a full tablet report...
I20260811 04:46:23.085673 7236 heartbeater.cc:503] Master 127.22.144.190:39221 was elected leader, sending a full tablet report...
I20260811 04:46:23.147210 7433 raft_consensus.cc:482] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:23.147476 7433 raft_consensus.cc:504] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.148644 7433 leader_election.cc:288] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
I20260811 04:46:23.152653 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.149061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:50441 (local address 127.22.144.131:44319)
0811 04:46:23.149317 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.149322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.149366 (+ 44us) server_negotiation.cc:407] Connection header received
0811 04:46:23.149600 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.149602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.149644 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.149703 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:23.150065 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.150562 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.151654 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.152071 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.152170 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.152247 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.152432 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.152516 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260811 04:46:23.152665 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.149061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:37887 (local address 127.22.144.130:34145)
0811 04:46:23.149216 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.149221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.149440 (+ 219us) server_negotiation.cc:407] Connection header received
0811 04:46:23.149594 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.149597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.149654 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.149747 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:23.150075 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.150561 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.151721 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.152102 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.152243 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.152295 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.152494 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.152566 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:46:23.152680 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.148944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34145 (local address 127.22.144.129:37887)
0811 04:46:23.149323 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.149337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:23.149454 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:23.149772 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:23.149775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:23.149815 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:23.149958 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.149963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.150670 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.150676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.151570 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.151580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.152253 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.152258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.152350 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.152368 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:46:23.152463 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 04:46:23.152523 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":93,"spinlock_wait_cycles":29440,"thread_start_us":110,"threads_started":1}
I20260811 04:46:23.152690 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.148944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44319 (local address 127.22.144.129:50441)
0811 04:46:23.149284 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.149296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:23.149442 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:23.149742 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:23.149745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:23.149762 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:23.149953 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.149959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.150725 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.150728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.151541 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:23.151549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.152201 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.152206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:23.152296 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.152328 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 04:46:23.152402 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 04:46:23.152486 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":30,"thread_start_us":131,"threads_started":1}
I20260811 04:46:23.153345 7186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b913f09fd9141e09b97c8e55be5d5f2" candidate_uuid: "f28491784a714619a4f2274a78752937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" is_pre_election: true
I20260811 04:46:23.153525 7348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b913f09fd9141e09b97c8e55be5d5f2" candidate_uuid: "f28491784a714619a4f2274a78752937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db1988ee561a4676a0f347629caebb92" is_pre_election: true
I20260811 04:46:23.153687 7186 raft_consensus.cc:2454] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f28491784a714619a4f2274a78752937 in term 0.
I20260811 04:46:23.153797 7348 raft_consensus.cc:2454] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f28491784a714619a4f2274a78752937 in term 0.
I20260811 04:46:23.154208 6956 leader_election.cc:302] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7ae5ee8ba6b34f24922a920cfc6c3fa5, db1988ee561a4676a0f347629caebb92, f28491784a714619a4f2274a78752937; no voters:
I20260811 04:46:23.154476 7433 raft_consensus.cc:2781] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:46:23.154562 7433 raft_consensus.cc:482] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:46:23.154606 7433 raft_consensus.cc:3037] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:23.155865 7433 raft_consensus.cc:504] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.156200 7433 leader_election.cc:288] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 1 election: Requested vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
I20260811 04:46:23.156531 7348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b913f09fd9141e09b97c8e55be5d5f2" candidate_uuid: "f28491784a714619a4f2274a78752937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db1988ee561a4676a0f347629caebb92"
I20260811 04:46:23.156524 7186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b913f09fd9141e09b97c8e55be5d5f2" candidate_uuid: "f28491784a714619a4f2274a78752937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
I20260811 04:46:23.156665 7348 raft_consensus.cc:3037] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:23.156656 7186 raft_consensus.cc:3037] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:23.157975 7186 raft_consensus.cc:2454] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f28491784a714619a4f2274a78752937 in term 1.
I20260811 04:46:23.157975 7348 raft_consensus.cc:2454] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f28491784a714619a4f2274a78752937 in term 1.
I20260811 04:46:23.158286 6956 leader_election.cc:302] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db1988ee561a4676a0f347629caebb92, f28491784a714619a4f2274a78752937; no voters:
I20260811 04:46:23.158455 7433 raft_consensus.cc:2781] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:23.158676 7433 raft_consensus.cc:686] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 LEADER]: Becoming Leader. State: Replica: f28491784a714619a4f2274a78752937, State: Running, Role: LEADER
I20260811 04:46:23.159010 7433 consensus_queue.cc:237] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:23.162361 6860 catalog_manager.cc:5495] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 reported cstate change: term changed from 0 to 1, leader changed from <none> to f28491784a714619a4f2274a78752937 (127.22.144.129). New cstate: current_term: 1 leader_uuid: "f28491784a714619a4f2274a78752937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } health_report { overall_health: HEALTHY } } }
I20260811 04:46:23.188302 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.184307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.22.144.129:43785)
0811 04:46:23.184437 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:23.184441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:23.184455 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:46:23.184531 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:23.184534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:23.184594 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:23.184690 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:46:23.184996 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.185557 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.186676 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:23.186911 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:23.187009 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:23.187321 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:46:23.187882 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:46:23.187987 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:23.188132 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 04:46:23.188190 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:46:23.194370 7186 raft_consensus.cc:1261] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Refusing update from remote peer f28491784a714619a4f2274a78752937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:23.194370 7348 raft_consensus.cc:1261] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Refusing update from remote peer f28491784a714619a4f2274a78752937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:23.195089 7433 consensus_queue.cc:1035] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:23.195287 7436 consensus_queue.cc:1035] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:23.202111 7439 mvcc.cc:205] Tried to move back new op lower bound from 7317190996756951040 to 7317190996622110720. Current Snapshot: MvccSnapshot[applied={T|T < 7317190996756951040}]
I20260811 04:46:23.203666 7441 mvcc.cc:205] Tried to move back new op lower bound from 7317190996756951040 to 7317190996622110720. Current Snapshot: MvccSnapshot[applied={T|T < 7317190996756951040}]
I20260811 04:46:23.213064 6860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:23.213249 6860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:23.215680 6860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Sending DeleteTablet for 3 replicas of tablet 8b913f09fd9141e09b97c8e55be5d5f2
I20260811 04:46:23.216621 7322 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:23 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:41562
I20260811 04:46:23.216677 7164 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:23 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:46706
I20260811 04:46:23.217175 7010 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:23 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:40210
I20260811 04:46:23.217609 7451 tablet_replica.cc:330] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: stopping tablet replica
I20260811 04:46:23.217952 7451 raft_consensus.cc:2229] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:46:23.218034 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 6933
I20260811 04:46:23.218323 7451 raft_consensus.cc:2258] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:23.220476 7449 tablet_replica.cc:330] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: stopping tablet replica
I20260811 04:46:23.220786 7449 raft_consensus.cc:2229] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:23.221101 7449 raft_consensus.cc:2258] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:23.222707 7449 ts_tablet_manager.cc:1905] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:23.225700 7450 tablet_replica.cc:330] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: stopping tablet replica
I20260811 04:46:23.226012 7450 raft_consensus.cc:2229] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:23.226241 7449 ts_tablet_manager.cc:1918] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:23.226356 7449 log.cc:1196] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/8b913f09fd9141e09b97c8e55be5d5f2
I20260811 04:46:23.226392 7450 raft_consensus.cc:2258] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:23.226742 7449 ts_tablet_manager.cc:1939] T 8b913f09fd9141e09b97c8e55be5d5f2 P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Deleting consensus metadata
I20260811 04:46:23.228186 6843 catalog_manager.cc:4841] TS 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): tablet 8b913f09fd9141e09b97c8e55be5d5f2 (table test-table [id=33bbe3e1040747058dd4af81cd7ae5cf]) successfully deleted
I20260811 04:46:23.228359 7450 ts_tablet_manager.cc:1905] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:23.230720 7450 ts_tablet_manager.cc:1918] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:23.230857 7450 log.cc:1196] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/8b913f09fd9141e09b97c8e55be5d5f2
I20260811 04:46:23.231208 7450 ts_tablet_manager.cc:1939] T 8b913f09fd9141e09b97c8e55be5d5f2 P db1988ee561a4676a0f347629caebb92: Deleting consensus metadata
I20260811 04:46:23.231887 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 7089
W20260811 04:46:23.231859 6846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:43785 (error 108)
I20260811 04:46:23.233103 6847 catalog_manager.cc:4841] TS db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): tablet 8b913f09fd9141e09b97c8e55be5d5f2 (table test-table [id=33bbe3e1040747058dd4af81cd7ae5cf]) successfully deleted
I20260811 04:46:23.238482 7414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:23.238075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43785 (local address 127.0.0.1:40222)
0811 04:46:23.238261 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:23.238365 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 04:46:23.238708 6846 catalog_manager.cc:4584] TS f28491784a714619a4f2274a78752937 (127.22.144.129:43785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b913f09fd9141e09b97c8e55be5d5f2: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43785: connect: Connection refused (error 111)
I20260811 04:46:23.240556 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 7242
I20260811 04:46:23.252504 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 6827
I20260811 04:46:43.262292 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39221
--webserver_interface=127.22.144.190
--webserver_port=37467
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:39221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:46:43.389568 7453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:43.389904 7453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:43.389950 7453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:43.394238 7453 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 04:46:43.394317 7453 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 04:46:43.394335 7453 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 04:46:43.394348 7453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:46:43.394373 7453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:43.394392 7453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:43.394472 7453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:46:43.394506 7453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:46:43.400441 7453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:39221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=37467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:43.401714 7453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:43.402871 7453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:43.419215 7464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:43.419227 7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:43.419292 7453 server_base.cc:1034] running on GCE node
W20260811 04:46:43.419237 7461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:43.420279 7453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:43.421700 7453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:43.422927 7453 hybrid_clock.cc:648] HybridClock initialized: now 1786423603422881 us; error 63 us; skew 500 ppm
Aug 11 04:46:43 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:43.426316 7453 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:46:43.427466 7453 webserver.cc:460] Webserver started at http://127.22.144.190:37467/ using document root <none> and password file <none>
I20260811 04:46:43.428066 7453 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:43.428144 7453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:43.433385 7453 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:43.436151 7473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:43.437844 7453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:46:43.438009 7453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "aa3023cb05d249179715be8e33d100a6"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "44e55db81da2eda31eb097ec37c2899f"
server_key_iv: "a4fe146cfe2f2e419ebc24640f9f8c6b"
server_key_version: "encryptionkey@0"
I20260811 04:46:43.438719 7453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:43.454594 7453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:43.458477 7453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:43.458822 7453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:43.467676 7453 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:39221
I20260811 04:46:43.468036 7536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:39221 every 8 connection(s)
I20260811 04:46:43.469075 7453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:46:43.471683 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 7453
I20260811 04:46:43.472214 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:43785
--local_ip_for_outbound_sockets=127.22.144.129
--tserver_master_addrs=127.22.144.190:39221
--webserver_port=32841
--webserver_interface=127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:46:43.474905 7537 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:46:43.477267 7537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap starting.
I20260811 04:46:43.503316 7537 log.cc:824] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:43.515100 7537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:46:43.515592 7537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap complete.
I20260811 04:46:43.521653 7537 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:43.522255 7537 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3023cb05d249179715be8e33d100a6, State: Initialized, Role: FOLLOWER
I20260811 04:46:43.522845 7537 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:43.523005 7537 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:46:43.523075 7537 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:46:43.523173 7537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 04:46:43.525942 7537 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:43.526377 7537 leader_election.cc:302] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3023cb05d249179715be8e33d100a6; no voters:
I20260811 04:46:43.526770 7537 leader_election.cc:288] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 04:46:43.526937 7542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 04:46:43.527271 7542 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 LEADER]: Becoming Leader. State: Replica: aa3023cb05d249179715be8e33d100a6, State: Running, Role: LEADER
I20260811 04:46:43.527665 7542 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:46:43.528168 7537 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:46:43.529399 7545 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3023cb05d249179715be8e33d100a6. Latest consensus state: current_term: 2 leader_uuid: "aa3023cb05d249179715be8e33d100a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } } }
I20260811 04:46:43.529569 7545 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:43.529400 7544 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa3023cb05d249179715be8e33d100a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } } }
I20260811 04:46:43.529983 7544 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:46:43.534075 7559 catalog_manager.cc:1259] Loaded cluster ID: b6de7b353ed54660ab299ff74c243784
I20260811 04:46:43.534145 7559 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: loading cluster ID for follower catalog manager: success
I20260811 04:46:43.536727 7559 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: acquiring CA information for follower catalog manager: success
I20260811 04:46:43.538370 7559 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:46:43.538767 7552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:46:43.541669 7552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:46:43.542200 7552 catalog_manager.cc:1259] Loaded cluster ID: b6de7b353ed54660ab299ff74c243784
I20260811 04:46:43.542263 7552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:46:43.543042 7552 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:46:43.543553 7552 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Loaded TSK: 0
I20260811 04:46:43.557333 7552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Generated new TSK 1
I20260811 04:46:43.557667 7552 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 04:46:43.620227 7540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:43.620502 7540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:43.620548 7540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:43.624938 7540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:43.625022 7540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:43.625087 7540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:46:43.631628 7540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:43785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=32841
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:43.632838 7540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:43.634073 7540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:43.646055 7569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:43.646059 7571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:43.646210 7567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:43.646497 7540 server_base.cc:1034] running on GCE node
I20260811 04:46:43.647027 7540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:43.647924 7540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:43.649160 7540 hybrid_clock.cc:648] HybridClock initialized: now 1786423603649131 us; error 58 us; skew 500 ppm
Aug 11 04:46:43 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:43.657014 7540 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:46:43.658396 7540 webserver.cc:460] Webserver started at http://127.22.144.129:32841/ using document root <none> and password file <none>
I20260811 04:46:43.659174 7540 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:43.659286 7540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:43.664381 7540 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 04:46:43.667328 7579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:43.668929 7540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:43.669073 7540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "f28491784a714619a4f2274a78752937"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "049db109d7222eeade027beece84116a"
server_key_iv: "e97fa7dc9fcb25edc4489c95becaece4"
server_key_version: "encryptionkey@0"
I20260811 04:46:43.669903 7540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:43.701309 7540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:43.705065 7540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:43.705245 7540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:43.705852 7540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:43.707008 7586 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 04:46:43.711653 7540 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 04:46:43.711807 7540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 04:46:43.711956 7540 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 04:46:43.714062 7540 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 04:46:43.714149 7540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260811 04:46:43.714501 7586 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Bootstrap starting.
I20260811 04:46:43.725234 7540 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:43785
I20260811 04:46:43.725337 7713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:43785 every 8 connection(s)
I20260811 04:46:43.726358 7540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:46:43.732822 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 7540
I20260811 04:46:43.733361 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:34145
--local_ip_for_outbound_sockets=127.22.144.130
--tserver_master_addrs=127.22.144.190:39221
--webserver_port=34797
--webserver_interface=127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:46:43.734534 7586 log.cc:824] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:46:43 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:43.744714 7586 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:46:43.745252 7586 tablet_bootstrap.cc:492] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Bootstrap complete.
I20260811 04:46:43.745784 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:43.728700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:52789)
0811 04:46:43.729309 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:43.729353 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:43.730442 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:43.731716 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:43.731724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:43.732175 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:43.732654 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:43.732669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.734321 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.734324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:43.735460 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:43.735474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.736481 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.736484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:43.736642 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:43.737289 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:43.737306 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:43.738833 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:43.741730 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:43.741737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:43.741763 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:43.742094 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:43.742573 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:43.742577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:43.742579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:43.742747 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:43.743634 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:43.743641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:43.743929 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 04:46:43.744298 (+ 369us) client_negotiation.cc:240] Negotiation successful
0811 04:46:43.744691 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":160,"threads_started":1}
I20260811 04:46:43.745769 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:43.728677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:52789 (local address 127.22.144.190:39221)
0811 04:46:43.729105 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:43.729125 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:43.729514 (+ 389us) server_negotiation.cc:407] Connection header received
0811 04:46:43.730905 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:43.730916 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:43.731216 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:43.731662 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:43.732832 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.734152 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.735608 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.736311 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.736502 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:43.739052 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:43.739059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:43.739077 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:43.739113 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:43.741397 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:43.742283 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:43.742292 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:43.742305 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:43.742449 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:43.742871 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:43.742876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:43.742878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:43.743495 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:43.743669 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:43.744362 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 04:46:43.744687 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":150,"threads_started":1}
I20260811 04:46:43.746961 7586 ts_tablet_manager.cc:1397] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Time spent bootstrapping tablet: real 0.033s user 0.023s sys 0.009s
I20260811 04:46:43.748839 7714 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39221
I20260811 04:46:43.749143 7714 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:43.749841 7714 heartbeater.cc:511] Master 127.22.144.190:39221 requested a full tablet report, sending...
I20260811 04:46:43.751133 7586 raft_consensus.cc:348] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:43.751717 7586 raft_consensus.cc:729] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28491784a714619a4f2274a78752937, State: Initialized, Role: FOLLOWER
I20260811 04:46:43.752260 7586 consensus_queue.cc:260] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:43.752991 7496 ts_manager.cc:194] Registered new tserver with Master: f28491784a714619a4f2274a78752937 (127.22.144.129:43785)
I20260811 04:46:43.753396 7586 ts_tablet_manager.cc:1428] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:46:43.757589 7496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:52789
I20260811 04:46:43.761047 7714 heartbeater.cc:503] Master 127.22.144.190:39221 was elected leader, sending a full tablet report...
Aug 11 04:46:43 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.129@KRBTEST.COM
I20260811 04:46:43.769157 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:43.756437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.22.144.129:43785)
0811 04:46:43.757265 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:43.757278 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:43.757305 (+ 27us) server_negotiation.cc:407] Connection header received
0811 04:46:43.757389 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:43.757394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:43.757683 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:43.757855 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:43.758678 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.759899 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.761282 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.761894 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.762060 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:43.764893 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:43.764905 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:43.764921 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:43.764981 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:43.766824 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:43.767445 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:43.767452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:43.767454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:43.767554 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:43.767890 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:43.767895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:43.767897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:43.768223 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:43.768386 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:43.768793 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 04:46:43.768997 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":103,"threads_started":1}
I20260811 04:46:43.769150 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:43.756323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43785 (local address 127.0.0.1:42460)
0811 04:46:43.756882 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:43.756948 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:43.757131 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:43.757896 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:43.757903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:43.758377 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:43.758514 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:43.758522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.760063 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.760068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:43.761114 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:43.761131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.762060 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.762064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:43.762229 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:43.762913 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:43.762936 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:43.764716 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:43.766973 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:43.766984 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:43.766990 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:43.767298 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:43.767679 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:43.767683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:43.767685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:43.767782 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:43.768373 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:43.768380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:43.768529 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 04:46:43.768695 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 04:46:43.768943 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":167,"threads_started":1}
I20260811 04:46:43.770354 7634 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b913f09fd9141e09b97c8e55be5d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:23 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260811 04:46:43.771317 7725 tablet_replica.cc:330] stopping tablet replica
I20260811 04:46:43.771679 7725 raft_consensus.cc:2229] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:43.771974 7725 raft_consensus.cc:2258] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:43.773064 7725 ts_tablet_manager.cc:1905] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:43.777730 7725 ts_tablet_manager.cc:1918] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:43.777841 7725 log.cc:1196] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal/wals/8b913f09fd9141e09b97c8e55be5d5f2
I20260811 04:46:43.778183 7725 ts_tablet_manager.cc:1939] T 8b913f09fd9141e09b97c8e55be5d5f2 P f28491784a714619a4f2274a78752937: Deleting consensus metadata
I20260811 04:46:43.780164 7476 catalog_manager.cc:4841] TS f28491784a714619a4f2274a78752937 (127.22.144.129:43785): tablet 8b913f09fd9141e09b97c8e55be5d5f2 (table test-table [id=33bbe3e1040747058dd4af81cd7ae5cf]) successfully deleted
W20260811 04:46:43.864264 7721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:43.864614 7721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:43.864703 7721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:43.868983 7721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:43.869105 7721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:43.869215 7721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:46:43.874852 7721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:34145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=34797
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:43.876084 7721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:43.877247 7721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:46:43.887979 7721 server_base.cc:1034] running on GCE node
W20260811 04:46:43.888022 7735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:43.887920 7732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:43.887920 7733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:43.888545 7721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:43.889336 7721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:43.890574 7721 hybrid_clock.cc:648] HybridClock initialized: now 1786423603890550 us; error 38 us; skew 500 ppm
Aug 11 04:46:43 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:43.893643 7721 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:46:43.894583 7721 webserver.cc:460] Webserver started at http://127.22.144.130:34797/ using document root <none> and password file <none>
I20260811 04:46:43.895206 7721 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:43.895301 7721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:43.899575 7721 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 04:46:43.901826 7742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:43.903307 7721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:46:43.903493 7721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "fc980b2b626e7580b93136400bc65b48"
server_key_iv: "6d1440fc35308f33f37843c5c0a12a9a"
server_key_version: "encryptionkey@0"
I20260811 04:46:43.904170 7721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:43.924537 7721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:43.928138 7721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:43.928414 7721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:43.929268 7721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:43.930404 7721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:43.930512 7721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:43.930579 7721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:43.930604 7721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:43.940845 7721 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:34145
I20260811 04:46:43.940922 7879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:34145 every 8 connection(s)
I20260811 04:46:43.942068 7721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:46:43.945650 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 7721
I20260811 04:46:43.946230 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:44319
--local_ip_for_outbound_sockets=127.22.144.131
--tserver_master_addrs=127.22.144.190:39221
--webserver_port=34497
--webserver_interface=127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 04:46:43 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423603, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:43.960183 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:43.944377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:38815 (local address 127.22.144.190:39221)
0811 04:46:43.944642 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:43.944646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:43.946241 (+ 1595us) server_negotiation.cc:407] Connection header received
0811 04:46:43.947367 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:43.947371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:43.947440 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:43.947523 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:43.949258 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.949908 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.951902 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.952236 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.952358 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:43.955971 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:43.955975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:43.955977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:43.956007 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:43.957867 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:43.958494 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:43.958498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:43.958502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:43.958589 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:43.958970 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:43.958975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:43.958976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:43.959180 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:43.959293 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:43.959862 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 04:46:43.960013 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 04:46:43.961090 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:43.944808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:38815)
0811 04:46:43.946066 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:43.946112 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:43.947121 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:43.947727 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:43.947735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:43.948275 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:43.949048 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:43.949072 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.950061 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.950067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:43.951691 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:43.951710 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:43.952460 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:43.952466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:43.952731 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:43.953555 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:43.953583 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:43.955733 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:43.958034 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:43.958043 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:43.958069 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:43.958358 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:43.958695 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:43.958698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:43.958700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:43.958859 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:43.959308 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:43.959317 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:43.959601 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 04:46:43.959809 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 04:46:43.960187 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":868,"threads_started":1}
I20260811 04:46:43.962687 7880 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39221
I20260811 04:46:43.962997 7880 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:43.963575 7880 heartbeater.cc:511] Master 127.22.144.190:39221 requested a full tablet report, sending...
I20260811 04:46:43.964771 7496 ts_manager.cc:194] Registered new tserver with Master: 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145)
I20260811 04:46:43.965618 7496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:38815
W20260811 04:46:44.076017 7883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:46:44.076329 7883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:46:44.076378 7883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:46:44.080674 7883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:46:44.080749 7883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:46:44.080813 7883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:46:44.086642 7883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:44319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=34497
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:46:44.087790 7883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:46:44.088814 7883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:46:44.099614 7888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:44.099611 7889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:46:44.099591 7891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:46:44.099876 7883 server_base.cc:1034] running on GCE node
I20260811 04:46:44.100310 7883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:46:44.101042 7883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:46:44.102260 7883 hybrid_clock.cc:648] HybridClock initialized: now 1786423604102225 us; error 48 us; skew 500 ppm
Aug 11 04:46:44 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423604, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:46:44.105659 7883 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:46:44.106734 7883 webserver.cc:460] Webserver started at http://127.22.144.131:34497/ using document root <none> and password file <none>
I20260811 04:46:44.107355 7883 fs_manager.cc:359] Metadata directory not provided
I20260811 04:46:44.107491 7883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:46:44.112059 7883 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 04:46:44.114499 7900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:44.115927 7883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:44.116089 7883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "db1988ee561a4676a0f347629caebb92"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "9c84b9318d865142df002da5fbb15b8b"
server_key_iv: "d5b15633105df615977e41f2ea3cafae"
server_key_version: "encryptionkey@0"
I20260811 04:46:44.116914 7883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:46:44.135663 7883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:46:44.139274 7883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:46:44.139523 7883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:46:44.140182 7883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:46:44.141069 7883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:46:44.141153 7883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:44.141224 7883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:46:44.141280 7883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:46:44.151727 7883 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:44319
I20260811 04:46:44.151751 8034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:44319 every 8 connection(s)
I20260811 04:46:44.152887 7883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:46:44.155982 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 7883
Aug 11 04:46:44 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423604, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:46:44.165033 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.156808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.22.144.190:39221)
0811 04:46:44.157103 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.157107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.157123 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:46:44.157204 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.157207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.157258 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.157361 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:44.158135 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.158701 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.159598 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.159853 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.159967 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.161453 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:44.161457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:44.161460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:44.161537 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:44.163231 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:44.163681 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:44.163686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:44.163688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:44.163775 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:44.164088 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:44.164092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:44.164093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:44.164275 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:44.164499 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.164699 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.164827 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":118,"threads_started":1}
I20260811 04:46:44.168635 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.154968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:48645 (local address 127.22.144.190:39221)
0811 04:46:44.155193 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.155197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.156014 (+ 817us) server_negotiation.cc:407] Connection header received
0811 04:46:44.157910 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.157914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.157964 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.158050 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:44.159372 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.159879 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.161234 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.161455 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.161625 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.164453 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:44.164456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:44.164459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:44.164489 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:44.166347 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:44.167034 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:44.167037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:44.167039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:44.167133 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:44.167509 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:44.167514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:44.167516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:44.167708 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:44.167827 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.168350 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.168475 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 04:46:44.169590 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.155307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:48645)
0811 04:46:44.155805 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:44.155849 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:44.157687 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:44.158284 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:44.158291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:44.158655 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:46:44.159215 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.159229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.160024 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.160030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.161090 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.161101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.161685 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.161689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.161934 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.162568 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:46:44.162591 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:46:44.164274 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:46:44.166446 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:44.166451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:44.166465 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:44.166891 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:44.167240 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:46:44.167243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:46:44.167245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:46:44.167400 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:46:44.167827 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:46:44.167835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:46:44.168100 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 04:46:44.168307 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 04:46:44.168680 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":105,"threads_started":1}
I20260811 04:46:44.170905 8035 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39221
I20260811 04:46:44.171128 8035 heartbeater.cc:464] Registering TS with master...
I20260811 04:46:44.171602 8035 heartbeater.cc:511] Master 127.22.144.190:39221 requested a full tablet report, sending...
I20260811 04:46:44.172647 7496 ts_manager.cc:194] Registered new tserver with Master: db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
I20260811 04:46:44.173198 7496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:48645
I20260811 04:46:44.179253 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:46:44.183578 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.179941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.22.144.190:39221)
0811 04:46:44.180095 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.180102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.180264 (+ 162us) server_negotiation.cc:407] Connection header received
0811 04:46:44.180404 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.180406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.180509 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.180599 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:46:44.180903 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.181428 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.182474 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.182693 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.182773 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.183058 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:46:44.183214 (+ 156us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 04:46:44.183430 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44984: token expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 04:46:44.183669 7717 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44984: token expired
W20260811 04:46:44.184060 7408 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.144.190:39221)
I20260811 04:46:44.192627 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.184421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.22.144.190:39221)
0811 04:46:44.184592 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.184595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.184607 (+ 12us) server_negotiation.cc:407] Connection header received
0811 04:46:44.184638 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.184640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.184676 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.184741 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:46:44.185365 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.185970 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.186784 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.187002 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.187096 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.189219 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:46:44.189223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:46:44.189226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:46:44.189260 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:46:44.191006 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:44.191456 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:44.191459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:44.191460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:44.191539 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:46:44.191849 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:46:44.191852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:46:44.191853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:46:44.192037 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:46:44.192174 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.192394 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.192497 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 04:46:44.195501 7496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:46:44.199011 7496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:46:44.207749 7634 tablet_service.cc:1467] Processing CreateTablet for tablet edac56ed14ce429d98183642563874cb (DEFAULT_TABLE table=test-table [id=b5952324cc2b40559883f1c4afffd0ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:44.208539 7634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edac56ed14ce429d98183642563874cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:44.211534 7586 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Bootstrap starting.
I20260811 04:46:44.212456 7586 tablet_bootstrap.cc:654] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:44.212759 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.206747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34145 (local address 127.0.0.1:52850)
0811 04:46:44.206923 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:44.206941 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:44.207031 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:44.208141 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:44.208144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:44.208162 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:44.208291 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.208298 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.209781 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.209787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.210863 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.210874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.212267 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.212271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.212377 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.212450 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 04:46:44.212524 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 04:46:44.212613 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 04:46:44.213253 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.206917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44319 (local address 127.0.0.1:49034)
0811 04:46:44.207227 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:44.207241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:44.207318 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:44.208295 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:44.208298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:44.208319 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:44.208841 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.208849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.210337 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.210340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.211341 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.211352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.212489 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.212496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.212596 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.212615 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 04:46:44.212671 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 04:46:44.212728 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":119,"threads_started":1}
I20260811 04:46:44.213354 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.207383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.22.144.131:44319)
0811 04:46:44.207778 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.207789 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.207810 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:46:44.207878 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.207882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.208121 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.208298 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:44.208984 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.210198 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.211485 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.212348 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.212856 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.213018 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.213071 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.213163 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":179,"threads_started":1}
I20260811 04:46:44.213908 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.206922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.22.144.130:34145)
0811 04:46:44.207526 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.207536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.207556 (+ 20us) server_negotiation.cc:407] Connection header received
0811 04:46:44.207677 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.207684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.207951 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.208433 (+ 482us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:44.208616 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.209663 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.211018 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.212131 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.212262 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.212423 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.213590 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.213727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":156,"threads_started":1}
I20260811 04:46:44.214179 7586 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: No bootstrap required, opened a new log
I20260811 04:46:44.214283 7586 ts_tablet_manager.cc:1397] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 04:46:44.214785 7586 raft_consensus.cc:348] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.214913 7586 raft_consensus.cc:374] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:44.214955 7586 raft_consensus.cc:729] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28491784a714619a4f2274a78752937, State: Initialized, Role: FOLLOWER
I20260811 04:46:44.215094 7586 consensus_queue.cc:260] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.215387 7586 ts_tablet_manager.cc:1428] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 04:46:44.216755 7956 tablet_service.cc:1467] Processing CreateTablet for tablet edac56ed14ce429d98183642563874cb (DEFAULT_TABLE table=test-table [id=b5952324cc2b40559883f1c4afffd0ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:44.216753 7796 tablet_service.cc:1467] Processing CreateTablet for tablet edac56ed14ce429d98183642563874cb (DEFAULT_TABLE table=test-table [id=b5952324cc2b40559883f1c4afffd0ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:46:44.217978 7956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edac56ed14ce429d98183642563874cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:44.217978 7796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edac56ed14ce429d98183642563874cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:46:44.226845 8051 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Bootstrap starting.
W20260811 04:46:44.227707 7716 tablet.cc:2369] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:46:44.227833 8052 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Bootstrap starting.
I20260811 04:46:44.229995 8051 tablet_bootstrap.cc:654] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:44.230680 8052 tablet_bootstrap.cc:654] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Neither blocks nor log segments found. Creating new log.
I20260811 04:46:44.231040 8051 log.cc:824] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:44.231525 8052 log.cc:824] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Log is configured to *not* fsync() on all Append() calls
I20260811 04:46:44.233103 8051 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: No bootstrap required, opened a new log
I20260811 04:46:44.233346 8052 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: No bootstrap required, opened a new log
I20260811 04:46:44.233352 8051 ts_tablet_manager.cc:1397] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 04:46:44.233557 8052 ts_tablet_manager.cc:1397] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:46:44.236362 8052 raft_consensus.cc:348] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.236362 8051 raft_consensus.cc:348] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.236660 8052 raft_consensus.cc:374] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:44.236718 8052 raft_consensus.cc:729] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db1988ee561a4676a0f347629caebb92, State: Initialized, Role: FOLLOWER
I20260811 04:46:44.236720 8051 raft_consensus.cc:374] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:46:44.236783 8051 raft_consensus.cc:729] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae5ee8ba6b34f24922a920cfc6c3fa5, State: Initialized, Role: FOLLOWER
I20260811 04:46:44.237251 8051 consensus_queue.cc:260] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.237227 8052 consensus_queue.cc:260] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.238013 8035 heartbeater.cc:503] Master 127.22.144.190:39221 was elected leader, sending a full tablet report...
I20260811 04:46:44.238227 7880 heartbeater.cc:503] Master 127.22.144.190:39221 was elected leader, sending a full tablet report...
I20260811 04:46:44.238337 8051 ts_tablet_manager.cc:1428] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260811 04:46:44.238338 8052 ts_tablet_manager.cc:1428] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
W20260811 04:46:44.403980 8036 tablet.cc:2369] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:46:44.416002 8055 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:46:44.416306 8055 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.417837 8055 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), f28491784a714619a4f2274a78752937 (127.22.144.129:43785)
I20260811 04:46:44.422947 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.418122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34145 (local address 127.22.144.131:59305)
0811 04:46:44.418790 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:44.418813 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:44.418947 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:44.419240 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:44.419242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:44.419259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:44.419353 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.419359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.420454 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.420457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.421686 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.421699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.422447 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.422451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.422559 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.422588 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 04:46:44.422688 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 04:46:44.422778 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":337,"spinlock_wait_cycles":8704,"thread_start_us":127,"threads_started":1}
I20260811 04:46:44.422923 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.418175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:59305 (local address 127.22.144.130:34145)
0811 04:46:44.418331 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.418335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.418956 (+ 621us) server_negotiation.cc:407] Connection header received
0811 04:46:44.419029 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.419034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.419095 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.419209 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:44.419489 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.420321 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.421859 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.422298 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.422447 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.422506 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.422707 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.422794 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 04:46:44.422999 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.418254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:38813 (local address 127.22.144.129:43785)
0811 04:46:44.418544 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.418547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.418562 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:46:44.418680 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.418683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.418757 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.418863 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:46:44.419219 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.420123 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.421859 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.422416 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.422578 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.422757 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.422801 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.422872 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260811 04:46:44.422998 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.418122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43785 (local address 127.22.144.131:38813)
0811 04:46:44.418251 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:44.418267 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:46:44.418371 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:46:44.418906 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:46:44.418908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:46:44.418938 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:46:44.419087 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.419093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.420242 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.420245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.421686 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:46:44.421699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.422568 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.422572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:46:44.422682 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.422703 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 04:46:44.422788 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 04:46:44.422865 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"spinlock_wait_cycles":384}
I20260811 04:46:44.423805 7821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edac56ed14ce429d98183642563874cb" candidate_uuid: "db1988ee561a4676a0f347629caebb92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" is_pre_election: true
I20260811 04:46:44.423799 7659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edac56ed14ce429d98183642563874cb" candidate_uuid: "db1988ee561a4676a0f347629caebb92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f28491784a714619a4f2274a78752937" is_pre_election: true
I20260811 04:46:44.424139 7659 raft_consensus.cc:2454] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db1988ee561a4676a0f347629caebb92 in term 0.
I20260811 04:46:44.424158 7821 raft_consensus.cc:2454] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db1988ee561a4676a0f347629caebb92 in term 0.
I20260811 04:46:44.424659 7904 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db1988ee561a4676a0f347629caebb92, f28491784a714619a4f2274a78752937; no voters:
I20260811 04:46:44.424999 8055 raft_consensus.cc:2781] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:46:44.425091 8055 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:46:44.425134 8055 raft_consensus.cc:3037] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:44.426251 8055 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.426616 8055 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [CANDIDATE]: Term 1 election: Requested vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), f28491784a714619a4f2274a78752937 (127.22.144.129:43785)
I20260811 04:46:44.427017 7659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edac56ed14ce429d98183642563874cb" candidate_uuid: "db1988ee561a4676a0f347629caebb92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f28491784a714619a4f2274a78752937"
I20260811 04:46:44.427052 7821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edac56ed14ce429d98183642563874cb" candidate_uuid: "db1988ee561a4676a0f347629caebb92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
I20260811 04:46:44.427146 7659 raft_consensus.cc:3037] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:44.427217 7821 raft_consensus.cc:3037] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:46:44.428328 7659 raft_consensus.cc:2454] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db1988ee561a4676a0f347629caebb92 in term 1.
I20260811 04:46:44.428426 7821 raft_consensus.cc:2454] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db1988ee561a4676a0f347629caebb92 in term 1.
I20260811 04:46:44.428656 7904 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db1988ee561a4676a0f347629caebb92, f28491784a714619a4f2274a78752937; no voters:
I20260811 04:46:44.428884 8055 raft_consensus.cc:2781] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:46:44.429136 8055 raft_consensus.cc:686] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 1 LEADER]: Becoming Leader. State: Replica: db1988ee561a4676a0f347629caebb92, State: Running, Role: LEADER
I20260811 04:46:44.429581 8055 consensus_queue.cc:237] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:46:44.433331 7496 catalog_manager.cc:5495] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 reported cstate change: term changed from 0 to 1, leader changed from <none> to db1988ee561a4676a0f347629caebb92 (127.22.144.131). New cstate: current_term: 1 leader_uuid: "db1988ee561a4676a0f347629caebb92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } health_report { overall_health: UNKNOWN } } }
W20260811 04:46:44.443445 7881 tablet.cc:2369] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:46:44.454747 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.450767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.22.144.131:44319)
0811 04:46:44.450903 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 04:46:44.450906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:46:44.450920 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:46:44.451076 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:46:44.451079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:46:44.451138 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:46:44.451261 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:46:44.451536 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.452038 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.452993 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:46:44.453225 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:46:44.453316 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:46:44.453807 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:46:44.454327 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:46:44.454426 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 04:46:44.454567 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 04:46:44.454625 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:46:44.461344 7659 raft_consensus.cc:1261] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Refusing update from remote peer db1988ee561a4676a0f347629caebb92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:44.461349 7821 raft_consensus.cc:1261] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Refusing update from remote peer db1988ee561a4676a0f347629caebb92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:46:44.462239 8059 consensus_queue.cc:1035] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:44.462503 8055 consensus_queue.cc:1035] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:46:44.469980 8063 mvcc.cc:205] Tried to move back new op lower bound from 7317191083865305088 to 7317191083746848768. Current Snapshot: MvccSnapshot[applied={T|T < 7317191083865305088}]
I20260811 04:46:44.470500 8062 mvcc.cc:205] Tried to move back new op lower bound from 7317191083865305088 to 7317191083746848768. Current Snapshot: MvccSnapshot[applied={T|T < 7317191083865305088}]
I20260811 04:46:44.480858 7496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:44.481043 7496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:46:44.483561 7496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Sending DeleteTablet for 3 replicas of tablet edac56ed14ce429d98183642563874cb
I20260811 04:46:44.484400 7796 tablet_service.cc:1514] Processing DeleteTablet for tablet edac56ed14ce429d98183642563874cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:44 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:52850
I20260811 04:46:44.484473 7956 tablet_service.cc:1514] Processing DeleteTablet for tablet edac56ed14ce429d98183642563874cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:44 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:49034
I20260811 04:46:44.484988 8075 tablet_replica.cc:330] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: stopping tablet replica
I20260811 04:46:44.484968 7634 tablet_service.cc:1514] Processing DeleteTablet for tablet edac56ed14ce429d98183642563874cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:46:44 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260811 04:46:44.485340 8075 raft_consensus.cc:2229] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:46:44.485731 8075 raft_consensus.cc:2258] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:44.485843 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 7540
I20260811 04:46:44.491199 8075 ts_tablet_manager.cc:1905] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:44.495472 8075 ts_tablet_manager.cc:1918] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:44.495625 8075 log.cc:1196] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/edac56ed14ce429d98183642563874cb
I20260811 04:46:44.496127 8075 ts_tablet_manager.cc:1939] T edac56ed14ce429d98183642563874cb P 7ae5ee8ba6b34f24922a920cfc6c3fa5: Deleting consensus metadata
I20260811 04:46:44.497778 8076 tablet_replica.cc:330] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: stopping tablet replica
I20260811 04:46:44.498113 8076 raft_consensus.cc:2229] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:46:44.498337 7474 catalog_manager.cc:4841] TS 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): tablet edac56ed14ce429d98183642563874cb (table test-table [id=b5952324cc2b40559883f1c4afffd0ed]) successfully deleted
I20260811 04:46:44.498564 8076 raft_consensus.cc:2258] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:46:44.499939 8076 ts_tablet_manager.cc:1905] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:46:44.502466 8076 ts_tablet_manager.cc:1918] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:46:44.502606 8076 log.cc:1196] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/edac56ed14ce429d98183642563874cb
I20260811 04:46:44.502930 8076 ts_tablet_manager.cc:1939] T edac56ed14ce429d98183642563874cb P db1988ee561a4676a0f347629caebb92: Deleting consensus metadata
I20260811 04:46:44.504186 7477 catalog_manager.cc:4841] TS db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): tablet edac56ed14ce429d98183642563874cb (table test-table [id=b5952324cc2b40559883f1c4afffd0ed]) successfully deleted
W20260811 04:46:44.505779 7476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:43785 (error 108)
I20260811 04:46:44.506340 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 7721
I20260811 04:46:44.507797 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:46:44.506640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43785 (local address 127.0.0.1:42476)
0811 04:46:44.507650 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 04:46:44.507717 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":912}
W20260811 04:46:44.508081 7476 catalog_manager.cc:4584] TS f28491784a714619a4f2274a78752937 (127.22.144.129:43785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet edac56ed14ce429d98183642563874cb: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43785: connect: Connection refused (error 111)
I20260811 04:46:44.515487 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 7883
I20260811 04:46:44.524415 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 7453
I20260811 04:47:04.535081 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39221
--webserver_interface=127.22.144.190
--webserver_port=37467
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:39221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:04.655486 8079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:04.655821 8079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:04.655893 8079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:04.660198 8079 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 04:47:04.660296 8079 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 04:47:04.660327 8079 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 04:47:04.660352 8079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:04.660395 8079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:04.660442 8079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:04.660476 8079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:04.660506 8079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:04.666010 8079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:39221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=37467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:04.667199 8079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:04.668294 8079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:47:04.678226 8079 server_base.cc:1034] running on GCE node
W20260811 04:47:04.678231 8086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:04.678261 8087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:04.678288 8090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:04.679100 8079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:04.680353 8079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:04.681578 8079 hybrid_clock.cc:648] HybridClock initialized: now 1786423624681567 us; error 29 us; skew 500 ppm
Aug 11 04:47:04 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423624, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:04.691980 8079 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:04.693118 8079 webserver.cc:460] Webserver started at http://127.22.144.190:37467/ using document root <none> and password file <none>
I20260811 04:47:04.693850 8079 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:04.693951 8079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:04.698364 8079 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:47:04.701241 8098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:04.702893 8079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 04:47:04.703076 8079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "aa3023cb05d249179715be8e33d100a6"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "44e55db81da2eda31eb097ec37c2899f"
server_key_iv: "a4fe146cfe2f2e419ebc24640f9f8c6b"
server_key_version: "encryptionkey@0"
I20260811 04:47:04.703843 8079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:04.726455 8079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:04.730623 8079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:04.730870 8079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:04.739424 8079 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:39221
I20260811 04:47:04.739765 8156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:39221 every 8 connection(s)
I20260811 04:47:04.740702 8079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:04.744284 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 8079
I20260811 04:47:04.744807 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:43785
--local_ip_for_outbound_sockets=127.22.144.129
--tserver_master_addrs=127.22.144.190:39221
--webserver_port=32841
--webserver_interface=127.22.144.129
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:47:04.745710 8157 sys_catalog.cc:263] Verifying existing consensus state
I20260811 04:47:04.748051 8157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap starting.
I20260811 04:47:04.767323 8157 log.cc:824] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:04.779229 8157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:47:04.779692 8157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Bootstrap complete.
I20260811 04:47:04.784130 8157 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:47:04.784587 8157 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3023cb05d249179715be8e33d100a6, State: Initialized, Role: FOLLOWER
I20260811 04:47:04.785115 8157 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:47:04.785272 8157 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:04.785364 8157 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:04.785517 8157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 04:47:04.787667 8157 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:47:04.788033 8157 leader_election.cc:302] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3023cb05d249179715be8e33d100a6; no voters:
I20260811 04:47:04.788429 8157 leader_election.cc:288] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 04:47:04.788523 8161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 04:47:04.788790 8161 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [term 3 LEADER]: Becoming Leader. State: Replica: aa3023cb05d249179715be8e33d100a6, State: Running, Role: LEADER
I20260811 04:47:04.789098 8161 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } }
I20260811 04:47:04.789794 8157 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:04.790205 8164 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3023cb05d249179715be8e33d100a6. Latest consensus state: current_term: 3 leader_uuid: "aa3023cb05d249179715be8e33d100a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } } }
I20260811 04:47:04.790335 8164 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:04.790597 8163 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "aa3023cb05d249179715be8e33d100a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3023cb05d249179715be8e33d100a6" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39221 } } }
I20260811 04:47:04.790705 8163 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:04.790872 8166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:04.796283 8166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:04.796998 8166 catalog_manager.cc:1259] Loaded cluster ID: b6de7b353ed54660ab299ff74c243784
I20260811 04:47:04.797062 8166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:04.799728 8166 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:04.800665 8166 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Loaded TSK: 0
I20260811 04:47:04.800788 8166 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa3023cb05d249179715be8e33d100a6: Loaded TSK: 1
F20260811 04:47:04.801802 8166 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f422b1119ad google::LogMessage::Fail() at ??:0
@ 0x7f422b11488a google::LogMessage::SendToLog() at ??:0
@ 0x7f422b1114de google::LogMessage::Flush() at ??:0
@ 0x7f422b112a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f422ddd518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f422ddcdf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4235a767bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4235a5099c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4235a51178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f4235a92df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4235ab182b std::function<>::operator()() at ??:0
@ 0x7f4235a50b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f4235a51f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4235a4e579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4235a92640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f422d3633d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f422d363c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f422d3653f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f422d354512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f422e9986db start_thread at ??:0
@ 0x7f4229c0f61f clone at ??:0
W20260811 04:47:04.892023 8159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:04.892339 8159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:04.892390 8159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:04.897104 8159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:04.897154 8159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:04.897217 8159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:04.903288 8159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:43785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=32841
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:04.904387 8159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:04.905608 8159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:04.915982 8211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:04.916081 8214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:04.915982 8212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:04.917171 8159 server_base.cc:1034] running on GCE node
I20260811 04:47:04.917659 8159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:04.918325 8159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:04.919518 8159 hybrid_clock.cc:648] HybridClock initialized: now 1786423624919483 us; error 45 us; skew 500 ppm
Aug 11 04:47:04 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423624, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:04.923475 8159 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:47:04.924830 8159 webserver.cc:460] Webserver started at http://127.22.144.129:32841/ using document root <none> and password file <none>
I20260811 04:47:04.925697 8159 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:04.925815 8159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:04.932070 8159 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:04.935428 8224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:04.937340 8159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:04.937542 8159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "f28491784a714619a4f2274a78752937"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "049db109d7222eeade027beece84116a"
server_key_iv: "e97fa7dc9fcb25edc4489c95becaece4"
server_key_version: "encryptionkey@0"
I20260811 04:47:04.938388 8159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:04.958021 8159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:04.962708 8159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:04.962960 8159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:04.963819 8159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:04.965353 8233 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 04:47:04.970080 8159 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 04:47:04.970181 8159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 04:47:04.970261 8159 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 04:47:04.972783 8159 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 04:47:04.972854 8159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260811 04:47:04.973305 8233 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Bootstrap starting.
I20260811 04:47:04.985371 8159 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:43785
I20260811 04:47:04.985806 8347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:43785 every 8 connection(s)
I20260811 04:47:04.986848 8159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:47:04.994199 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 8159
I20260811 04:47:04.994791 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:34145
--local_ip_for_outbound_sockets=127.22.144.130
--tserver_master_addrs=127.22.144.190:39221
--webserver_port=34797
--webserver_interface=127.22.144.130
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:47:04.996554 8233 log.cc:824] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:05.004820 8233 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 04:47:05.005295 8233 tablet_bootstrap.cc:492] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Bootstrap complete.
I20260811 04:47:05.006633 8233 ts_tablet_manager.cc:1397] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.012s
I20260811 04:47:05.009330 8233 raft_consensus.cc:348] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:05.010084 8233 raft_consensus.cc:729] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28491784a714619a4f2274a78752937, State: Initialized, Role: FOLLOWER
I20260811 04:47:05.010721 8233 consensus_queue.cc:260] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:05.011822 8233 ts_tablet_manager.cc:1428] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260811 04:47:05.120193 8350 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:47:04.989562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:44307)
0811 04:47:04.990283 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:04.990359 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:04.991626 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:05.119147 (+127521us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":205,"threads_started":1}
W20260811 04:47:05.120923 8348 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:39221 (0 consecutive failures): Network error: Failed to ping master at 127.22.144.190:39221: Client connection negotiation failed: client connection to 127.22.144.190:39221: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 04:47:05.121771 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.121432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:34311)
0811 04:47:05.121610 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.121697 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 04:47:05.122586 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.122340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:36085)
0811 04:47:05.122464 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.122523 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 04:47:05.122830 8348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 04:47:05.149576 8352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:05.149883 8352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:05.149931 8352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:05.154098 8352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:05.154198 8352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:05.154320 8352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:05.159848 8352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:34145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=34797
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:05.161000 8352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:05.162027 8352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:05.172441 8365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:05.172464 8362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:05.172523 8352 server_base.cc:1034] running on GCE node
W20260811 04:47:05.172523 8363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:05.173275 8352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:05.174057 8352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:05.175269 8352 hybrid_clock.cc:648] HybridClock initialized: now 1786423625175230 us; error 51 us; skew 500 ppm
Aug 11 04:47:05 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423625, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:05.178721 8352 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:47:05.179790 8352 webserver.cc:460] Webserver started at http://127.22.144.130:34797/ using document root <none> and password file <none>
I20260811 04:47:05.180416 8352 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:05.180490 8352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:05.185022 8352 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:05.187731 8376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:05.189347 8352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:05.189596 8352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "fc980b2b626e7580b93136400bc65b48"
server_key_iv: "6d1440fc35308f33f37843c5c0a12a9a"
server_key_version: "encryptionkey@0"
I20260811 04:47:05.190272 8352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:05.210806 8352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:05.214357 8352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:05.214608 8352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:05.215318 8352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:05.216250 8352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:05.216364 8352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:05.216447 8352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:05.216499 8352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:05.228120 8352 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:34145
I20260811 04:47:05.228143 8505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:34145 every 8 connection(s)
I20260811 04:47:05.229362 8352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:05.234122 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.232030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:44725)
0811 04:47:05.232618 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.232748 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":139,"threads_started":1}
W20260811 04:47:05.234793 8506 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:39221 (0 consecutive failures): Network error: Failed to ping master at 127.22.144.190:39221: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
I20260811 04:47:05.235635 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.235333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:47557)
0811 04:47:05.235500 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.235559 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 04:47:05.236099 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 8352
I20260811 04:47:05.236531 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.236262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:42057)
0811 04:47:05.236390 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.236453 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 04:47:05.236752 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:44319
--local_ip_for_outbound_sockets=127.22.144.131
--tserver_master_addrs=127.22.144.190:39221
--webserver_port=34497
--webserver_interface=127.22.144.131
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:05.236779 8506 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 04:47:05.360785 8509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:05.361102 8509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:05.361151 8509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:05.365461 8509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:05.365621 8509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:05.365706 8509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:05.371712 8509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:44319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=34497
--tserver_master_addrs=127.22.144.190:39221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:05.372889 8509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:05.374120 8509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:05.385735 8514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:05.385691 8516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:05.385702 8518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:05.385900 8509 server_base.cc:1034] running on GCE node
I20260811 04:47:05.386476 8509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:05.387230 8509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:05.388456 8509 hybrid_clock.cc:648] HybridClock initialized: now 1786423625388425 us; error 44 us; skew 500 ppm
Aug 11 04:47:05 dist-test-slave-ss4n krb5kdc[6809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423625, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:05.391562 8509 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:47:05.392577 8509 webserver.cc:460] Webserver started at http://127.22.144.131:34497/ using document root <none> and password file <none>
I20260811 04:47:05.393146 8509 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:05.393210 8509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:05.397599 8509 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:47:05.399883 8528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:05.401288 8509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:05.401397 8509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "db1988ee561a4676a0f347629caebb92"
format_stamp: "Formatted at 2026-08-11 04:46:22 on dist-test-slave-ss4n"
server_key: "9c84b9318d865142df002da5fbb15b8b"
server_key_iv: "d5b15633105df615977e41f2ea3cafae"
server_key_version: "encryptionkey@0"
I20260811 04:47:05.402073 8509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:05.416795 8509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:05.420445 8509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:05.420681 8509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:05.421309 8509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:05.422289 8509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:05.422365 8509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:05.422434 8509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:05.422485 8509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:05.432896 8509 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:44319
I20260811 04:47:05.432914 8659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:44319 every 8 connection(s)
I20260811 04:47:05.434125 8509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:05.435907 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 8509
I20260811 04:47:05.438571 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.436564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:44837)
0811 04:47:05.437194 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.437306 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":159,"threads_started":1}
W20260811 04:47:05.439038 8660 heartbeater.cc:644] Failed to heartbeat to 127.22.144.190:39221 (0 consecutive failures): Network error: Failed to ping master at 127.22.144.190:39221: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
I20260811 04:47:05.439692 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.439462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:46481)
0811 04:47:05.439583 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.439637 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 04:47:05.440475 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:05.440207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:48253)
0811 04:47:05.440361 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:05.440414 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 04:47:05.440709 8660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 04:47:06.124207 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.123696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:44091)
0811 04:47:06.124024 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:06.124096 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":87,"threads_started":1}
I20260811 04:47:06.237980 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.237430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:57443)
0811 04:47:06.237812 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:06.237875 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":188,"threads_started":1}
I20260811 04:47:06.441977 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.441389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:38185)
0811 04:47:06.441771 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:06.441841 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":155,"threads_started":1}
I20260811 04:47:06.829216 8668 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:06.829560 8668 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:06.831072 8668 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
Aug 11 04:47:06 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423624, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.130@KRBTEST.COM
Aug 11 04:47:06 dist-test-slave-ss4n krb5kdc[6809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423624, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.131@KRBTEST.COM
I20260811 04:47:06.847617 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.831821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:50377 (local address 127.22.144.131:44319)
0811 04:47:06.832222 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:06.832254 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:06.832303 (+ 49us) server_negotiation.cc:407] Connection header received
0811 04:47:06.833050 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:06.833057 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:06.833431 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:06.833843 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:06.835402 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.836442 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.838062 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.838596 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.838785 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:06.842292 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:06.842298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:06.842312 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:06.842349 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:06.844497 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:06.845309 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:06.845318 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:06.845328 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:06.845460 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:06.845954 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:06.845957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:06.845959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:06.846405 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:06.846513 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:06.847141 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 04:47:06.847411 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":70,"threads_started":1}
I20260811 04:47:06.847604 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.831392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:34145 (local address 127.22.144.129:52533)
0811 04:47:06.832209 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:06.832224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:06.832348 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:06.833928 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:06.833935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:06.834582 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:06.835170 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:06.835188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.836705 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.836708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:06.837898 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:06.837912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.838764 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.838768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:06.839034 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:06.839818 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:06.839836 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:06.841583 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:06.844894 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:06.844897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:06.844899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:06.845196 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:06.845724 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:06.845731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:06.845733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:06.845835 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:06.846579 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:06.846582 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:06.846803 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 04:47:06.846998 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 04:47:06.847370 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":723,"spinlock_wait_cycles":8064,"thread_start_us":87,"threads_started":1}
I20260811 04:47:06.847738 8669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.831392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44319 (local address 127.22.144.129:50377)
0811 04:47:06.831776 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:06.831794 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:06.832023 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:06.833930 (+ 1907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:06.833936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:06.834583 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:06.835175 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:06.835189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.836641 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.836649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:06.837898 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:06.837912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.838750 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.838754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:06.839038 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:06.839882 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:06.839896 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:06.842040 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:06.844690 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:06.844699 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:06.844720 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:06.845185 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:06.845591 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:06.845598 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:06.845604 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:06.845833 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:06.846540 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:06.846549 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:06.846805 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 04:47:06.847008 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 04:47:06.847470 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":277,"spinlock_wait_cycles":35584,"thread_start_us":81,"threads_started":1}
I20260811 04:47:06.848356 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:06.831603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:52533 (local address 127.22.144.130:34145)
0811 04:47:06.832189 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:06.832205 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:06.832343 (+ 138us) server_negotiation.cc:407] Connection header received
0811 04:47:06.833100 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:06.833107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:06.833432 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:06.833843 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:06.835400 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.836499 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.838076 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:06.838600 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:06.838801 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:06.841941 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:06.841951 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:06.841973 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:06.842071 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:06.844768 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:06.845342 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:06.845349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:06.845355 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:06.845563 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:06.845937 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:06.845944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:06.845949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:06.846424 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:06.846598 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:06.847133 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 04:47:06.847474 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":105,"threads_started":1}
W20260811 04:47:06.850409 8228 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
W20260811 04:47:06.850590 8225 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
I20260811 04:47:06.850703 8225 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f28491784a714619a4f2274a78752937; no voters: 7ae5ee8ba6b34f24922a920cfc6c3fa5, db1988ee561a4676a0f347629caebb92
I20260811 04:47:06.851037 8668 raft_consensus.cc:2726] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:47:07.125607 8669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:07.125228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:46273)
0811 04:47:07.125375 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:07.125434 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 04:47:07.239307 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:07.238786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:49171)
0811 04:47:07.239107 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:07.239178 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":80,"threads_started":1}
I20260811 04:47:07.443406 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:07.442929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:36377)
0811 04:47:07.443235 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:07.443302 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":75,"threads_started":1}
I20260811 04:47:08.126878 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:08.126430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:37253)
0811 04:47:08.126697 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:08.126760 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":81,"threads_started":1}
I20260811 04:47:08.240731 8676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:08.240229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:56429)
0811 04:47:08.240555 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:08.240626 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":100,"threads_started":1}
I20260811 04:47:08.444990 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:08.444448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:38527)
0811 04:47:08.444807 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:08.444875 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":86,"threads_started":1}
I20260811 04:47:08.445631 8678 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:08.445779 8678 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:08.446236 8678 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
W20260811 04:47:08.447170 8225 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
W20260811 04:47:08.447285 8228 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
I20260811 04:47:08.447422 8228 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f28491784a714619a4f2274a78752937; no voters: 7ae5ee8ba6b34f24922a920cfc6c3fa5, db1988ee561a4676a0f347629caebb92
I20260811 04:47:08.447647 8678 raft_consensus.cc:2726] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:47:09.128342 8679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:09.127829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:47757)
0811 04:47:09.128154 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:09.128222 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":89,"threads_started":1}
I20260811 04:47:09.242362 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:09.241705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:56205)
0811 04:47:09.242101 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:09.242206 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260811 04:47:09.446609 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:09.445994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:34831)
0811 04:47:09.446395 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:09.446463 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":143,"threads_started":1}
I20260811 04:47:10.130046 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:10.129459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:35325)
0811 04:47:10.129880 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:10.129942 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":107,"threads_started":1}
I20260811 04:47:10.243904 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:10.243387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:57957)
0811 04:47:10.243737 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:10.243803 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":151,"threads_started":1}
I20260811 04:47:10.448033 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:10.447615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:38597)
0811 04:47:10.447879 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:10.447935 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260811 04:47:11.131438 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:11.130988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:57371)
0811 04:47:11.131280 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:11.131341 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":95,"threads_started":1}
I20260811 04:47:11.245419 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:11.244899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:42241)
0811 04:47:11.245255 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:11.245320 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":84,"threads_started":1}
I20260811 04:47:11.408731 8692 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:11.408867 8692 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:11.409364 8692 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
W20260811 04:47:11.410269 8225 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
W20260811 04:47:11.410369 8228 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
I20260811 04:47:11.410488 8228 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f28491784a714619a4f2274a78752937; no voters: 7ae5ee8ba6b34f24922a920cfc6c3fa5, db1988ee561a4676a0f347629caebb92
I20260811 04:47:11.410651 8692 raft_consensus.cc:2726] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:47:11.449455 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:11.448965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:38163)
0811 04:47:11.449292 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:11.449354 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":101,"threads_started":1}
I20260811 04:47:12.132841 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:12.132342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:51033)
0811 04:47:12.132663 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:12.132732 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":87,"threads_started":1}
I20260811 04:47:12.246793 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:12.246273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:59403)
0811 04:47:12.246610 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:12.246695 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":77,"threads_started":1}
I20260811 04:47:12.450848 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:12.450373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:50763)
0811 04:47:12.450685 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:12.450748 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":73,"threads_started":1}
I20260811 04:47:13.134280 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:13.133828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:33781)
0811 04:47:13.134118 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:13.134183 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260811 04:47:13.204397 8698 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:13.204551 8698 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:13.205080 8698 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
W20260811 04:47:13.206003 8225 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
W20260811 04:47:13.206130 8228 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
I20260811 04:47:13.206228 8228 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f28491784a714619a4f2274a78752937; no voters: 7ae5ee8ba6b34f24922a920cfc6c3fa5, db1988ee561a4676a0f347629caebb92
I20260811 04:47:13.206408 8698 raft_consensus.cc:2726] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:47:13.248198 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:13.247732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:51731)
0811 04:47:13.248012 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:13.248097 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260811 04:47:13.452211 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:13.451737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:60001)
0811 04:47:13.452027 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:13.452106 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
I20260811 04:47:14.135601 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:14.135179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:53451)
0811 04:47:14.135433 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:14.135491 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
I20260811 04:47:14.249799 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:14.249236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:46109)
0811 04:47:14.249617 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:14.249689 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":91,"threads_started":1}
I20260811 04:47:14.453707 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:14.453125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:34485)
0811 04:47:14.453416 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:14.453583 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":82,"threads_started":1}
I20260811 04:47:15.136912 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:15.136484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:41859)
0811 04:47:15.136756 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:15.136815 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
I20260811 04:47:15.251286 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:15.250805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:47129)
0811 04:47:15.251121 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:15.251188 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":108,"threads_started":1}
I20260811 04:47:15.455197 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:15.454616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:42941)
0811 04:47:15.454955 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:15.455043 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
I20260811 04:47:16.138198 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:16.137739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:43595)
0811 04:47:16.138030 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:16.138094 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":76,"threads_started":1}
I20260811 04:47:16.252787 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:16.252279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:55431)
0811 04:47:16.252605 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:16.252676 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":82,"threads_started":1}
I20260811 04:47:16.456820 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:16.456285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:55683)
0811 04:47:16.456629 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:16.456702 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":91,"threads_started":1}
I20260811 04:47:17.139744 8712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:17.139161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:54617)
0811 04:47:17.139555 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:17.139621 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":133,"threads_started":1}
I20260811 04:47:17.254287 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:17.253810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:38221)
0811 04:47:17.254093 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:17.254189 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
I20260811 04:47:17.458243 8714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:17.457803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:52027)
0811 04:47:17.458076 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:17.458142 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260811 04:47:18.141211 8715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:18.140728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:50575)
0811 04:47:18.141003 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:18.141063 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260811 04:47:18.255632 8716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:18.255075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:32849)
0811 04:47:18.255391 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:18.255459 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
I20260811 04:47:18.459641 8718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:18.459153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:58565)
0811 04:47:18.459463 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:18.459531 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260811 04:47:19.142576 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:19.142109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:53161)
0811 04:47:19.142412 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:19.142476 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":101,"threads_started":1}
I20260811 04:47:19.180743 8722 raft_consensus.cc:482] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:19.180927 8722 raft_consensus.cc:504] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ae5ee8ba6b34f24922a920cfc6c3fa5" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 34145 } } peers { permanent_uuid: "db1988ee561a4676a0f347629caebb92" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44319 } } peers { permanent_uuid: "f28491784a714619a4f2274a78752937" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43785 } }
I20260811 04:47:19.181458 8722 leader_election.cc:288] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145), db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319)
W20260811 04:47:19.182440 8225 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ae5ee8ba6b34f24922a920cfc6c3fa5 (127.22.144.130:34145): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
W20260811 04:47:19.182551 8228 leader_election.cc:341] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db1988ee561a4676a0f347629caebb92 (127.22.144.131:44319): Not found: Tablet not found: edac56ed14ce429d98183642563874cb
I20260811 04:47:19.182682 8228 leader_election.cc:302] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f28491784a714619a4f2274a78752937; no voters: 7ae5ee8ba6b34f24922a920cfc6c3fa5, db1988ee561a4676a0f347629caebb92
I20260811 04:47:19.182888 8722 raft_consensus.cc:2726] T edac56ed14ce429d98183642563874cb P f28491784a714619a4f2274a78752937 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 04:47:19.256973 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:19.256537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:47001)
0811 04:47:19.256809 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:19.256872 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260811 04:47:19.461071 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:19.460611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.131:41603)
0811 04:47:19.460903 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:19.460969 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":87,"threads_started":1}
I20260811 04:47:20.144019 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:20.143609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.129:57137)
0811 04:47:20.143865 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:20.143923 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260811 04:47:20.258452 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:20.258025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39221 (local address 127.22.144.130:50695)
0811 04:47:20.258293 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:20.258356 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.190:39221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260811 04:47:20.440591 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 8159
I20260811 04:47:20.451381 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 8352
I20260811 04:47:20.459722 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 8509
2026-08-11T04:47:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59738 ms)
[----------] 30 tests from SecurityITest (365919 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:20 dist-test-slave-ss4n krb5kdc[8729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:20 dist-test-slave-ss4n krb5kdc[8729](info): set up 2 sockets
Aug 11 04:47:20 dist-test-slave-ss4n krb5kdc[8729](info): commencing operation
krb5kdc: starting...
W20260811 04:47:21.574780 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:21 dist-test-slave-ss4n krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:21.717748 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39763
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:39763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:21.839279 8747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:21.839623 8747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:21.839668 8747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:21.843837 8747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:21.843906 8747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:21.843927 8747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:21.843945 8747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:21.843961 8747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:21.849714 8747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:39763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:21.850875 8747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:21.851965 8747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:21.862011 8752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:21.862071 8747 server_base.cc:1034] running on GCE node
W20260811 04:47:21.862182 8755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:21.861963 8753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:21.862860 8747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:21.864198 8747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:21.865420 8747 hybrid_clock.cc:648] HybridClock initialized: now 1786423641865400 us; error 35 us; skew 500 ppm
Aug 11 04:47:21 dist-test-slave-ss4n krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423641, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:21.868505 8747 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:21.869661 8747 webserver.cc:460] Webserver started at http://127.22.144.190:40955/ using document root <none> and password file <none>
I20260811 04:47:21.870314 8747 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:21.870450 8747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:21.870711 8747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:21.872989 8747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9"
format_stamp: "Formatted at 2026-08-11 04:47:21 on dist-test-slave-ss4n"
server_key: "2731936ccb5f89493265218da8a8e1ee"
server_key_iv: "d3f68d728e551f7cb90ba29a85082db1"
server_key_version: "encryptionkey@0"
I20260811 04:47:21.873690 8747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9"
format_stamp: "Formatted at 2026-08-11 04:47:21 on dist-test-slave-ss4n"
server_key: "2731936ccb5f89493265218da8a8e1ee"
server_key_iv: "d3f68d728e551f7cb90ba29a85082db1"
server_key_version: "encryptionkey@0"
I20260811 04:47:21.877851 8747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:47:21.880409 8763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:21.881816 8747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:47:21.881983 8747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9"
format_stamp: "Formatted at 2026-08-11 04:47:21 on dist-test-slave-ss4n"
server_key: "2731936ccb5f89493265218da8a8e1ee"
server_key_iv: "d3f68d728e551f7cb90ba29a85082db1"
server_key_version: "encryptionkey@0"
I20260811 04:47:21.882107 8747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:21.906128 8747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:21.909780 8747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:21.910075 8747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:21.920501 8747 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:39763
I20260811 04:47:21.920675 8825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:39763 every 8 connection(s)
I20260811 04:47:21.922073 8747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:21.925366 8827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:21.925930 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 8747
I20260811 04:47:21.926087 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:21.926451 23106 external_mini_cluster.cc:1428] Setting key 0d1bb946e175a363184f0ba78282cbc4
I20260811 04:47:21.932742 8827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9: Bootstrap starting.
I20260811 04:47:21.935853 8827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:21.936851 8827 log.cc:824] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:47:21 dist-test-slave-ss4n krb5kdc[8729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:21.939710 8827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9: No bootstrap required, opened a new log
I20260811 04:47:21.944029 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:21.928124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.22.144.190:39763)
0811 04:47:21.928643 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:21.928664 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:21.928705 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:47:21.929466 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:21.929569 (+ 103us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:21.929953 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:21.930285 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:21.931358 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:21.932526 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:21.933796 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:21.934309 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:21.934442 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:21.938020 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:21.938027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:21.938046 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:21.938079 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:21.940384 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:21.940905 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:21.940912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:21.940918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:21.941068 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:21.941405 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:21.941409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:21.941411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:21.942021 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:21.942289 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:21.942582 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 04:47:21.942970 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":175,"threads_started":1}
I20260811 04:47:21.944984 8827 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39763 } }
I20260811 04:47:21.945326 8827 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:21.945406 8827 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae96bbe0d50a4df0a6ec3e2b2a6fb5b9, State: Initialized, Role: FOLLOWER
I20260811 04:47:21.945947 8827 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39763 } }
I20260811 04:47:21.946130 8827 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:21.946205 8827 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:21.946308 8827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:21.947315 8827 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39763 } }
I20260811 04:47:21.947696 8827 leader_election.cc:302] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae96bbe0d50a4df0a6ec3e2b2a6fb5b9; no voters:
I20260811 04:47:21.948042 8827 leader_election.cc:288] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:21.948220 8832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:21.948560 8832 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [term 1 LEADER]: Becoming Leader. State: Replica: ae96bbe0d50a4df0a6ec3e2b2a6fb5b9, State: Running, Role: LEADER
I20260811 04:47:21.948911 8832 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39763 } }
I20260811 04:47:21.949568 8827 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:21.951058 8834 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae96bbe0d50a4df0a6ec3e2b2a6fb5b9. Latest consensus state: current_term: 1 leader_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39763 } } }
I20260811 04:47:21.951068 8833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae96bbe0d50a4df0a6ec3e2b2a6fb5b9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39763 } } }
I20260811 04:47:21.951195 8834 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:21.951218 8833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:21.952065 8841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:21.955202 8841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:21.961745 8841 catalog_manager.cc:1347] Generated new cluster ID: 1e5affb27188408ead78e7e5fdaeefd9
I20260811 04:47:21.961844 8841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:21.976302 8841 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:21.978094 8841 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:21.989102 8841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9: Generated new TSK 0
I20260811 04:47:21.989915 8841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:47:22.030712 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39763
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:22.158840 8856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:22.159119 8856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:22.159165 8856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:22.163383 8856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:22.163452 8856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:22.163549 8856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:22.169067 8856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:22.170192 8856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:22.171291 8856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:22.181255 8863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:22.181321 8864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:22.181432 8856 server_base.cc:1034] running on GCE node
W20260811 04:47:22.181339 8866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:22.182011 8856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:22.182691 8856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:22.183888 8856 hybrid_clock.cc:648] HybridClock initialized: now 1786423642183870 us; error 38 us; skew 500 ppm
Aug 11 04:47:22 dist-test-slave-ss4n krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:22.187078 8856 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:47:22.187973 8856 webserver.cc:460] Webserver started at http://127.22.144.129:35345/ using document root <none> and password file <none>
I20260811 04:47:22.188561 8856 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:22.188629 8856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:22.188894 8856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:22.191104 8856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "0baa556568404ad5aecec132807c09c4"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "4acae98b7d1509401f1490eaa484e8a5"
server_key_iv: "f7809b3bb14b1c0caeae1ca133deca51"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.191777 8856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "0baa556568404ad5aecec132807c09c4"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "4acae98b7d1509401f1490eaa484e8a5"
server_key_iv: "f7809b3bb14b1c0caeae1ca133deca51"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.195737 8856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:22.198282 8874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.199489 8856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:47:22.199635 8856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "0baa556568404ad5aecec132807c09c4"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "4acae98b7d1509401f1490eaa484e8a5"
server_key_iv: "f7809b3bb14b1c0caeae1ca133deca51"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.199743 8856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:22.223006 8856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:22.226764 8856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:22.227002 8856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:22.227686 8856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:22.228672 8856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:22.228745 8856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.228821 8856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:22.228874 8856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.238248 8856 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:33015
I20260811 04:47:22.238281 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:33015 every 8 connection(s)
I20260811 04:47:22.239251 8856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:47:22.240006 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 8856
I20260811 04:47:22.240161 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:47:22.240522 23106 external_mini_cluster.cc:1428] Setting key 60e0c3a1573f236a353ebac08eaec28f
Aug 11 04:47:22 dist-test-slave-ss4n krb5kdc[8729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
I20260811 04:47:22.256752 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.241312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:39551 (local address 127.22.144.190:39763)
0811 04:47:22.241454 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.241457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.242676 (+ 1219us) server_negotiation.cc:407] Connection header received
0811 04:47:22.244839 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.244844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.244910 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.245013 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:22.246349 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.246928 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.248820 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.249052 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.249165 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.252434 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:22.252438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:22.252442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:22.252476 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:22.254229 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.254837 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.254841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.254842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.254920 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.255363 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.255366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.255368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.255572 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:22.255704 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.256321 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.256500 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:47:22.257797 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.241868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39763 (local address 127.22.144.129:39551)
0811 04:47:22.242511 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.242577 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.244551 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.245207 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.245214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.245680 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:22.246195 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.246210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.247080 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.247086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.248690 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.248705 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.249278 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.249284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.249578 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.250332 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:22.250355 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:22.252219 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:22.254366 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:22.254374 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:22.254392 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:22.254708 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:22.255023 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:22.255026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:22.255028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:22.255245 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:22.255705 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:22.255713 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:22.255998 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.256211 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.256571 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":198,"threads_started":1}
I20260811 04:47:22.259258 9015 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39763
I20260811 04:47:22.259524 9015 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:22.260066 9015 heartbeater.cc:511] Master 127.22.144.190:39763 requested a full tablet report, sending...
I20260811 04:47:22.261960 8784 ts_manager.cc:194] Registered new tserver with Master: 0baa556568404ad5aecec132807c09c4 (127.22.144.129:33015)
I20260811 04:47:22.263756 8784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:39551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:47:22.276849 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39763
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:22.395113 9020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:22.395403 9020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:22.395483 9020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:22.399720 9020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:22.399791 9020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:22.399893 9020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:22.405788 9020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:22.406908 9020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:22.408022 9020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:22.418066 9025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:22.418051 9029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:22.418076 9026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:22.418695 9020 server_base.cc:1034] running on GCE node
I20260811 04:47:22.419145 9020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:22.419869 9020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:22.421051 9020 hybrid_clock.cc:648] HybridClock initialized: now 1786423642421042 us; error 37 us; skew 500 ppm
Aug 11 04:47:22 dist-test-slave-ss4n krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:22.424181 9020 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:47:22.425215 9020 webserver.cc:460] Webserver started at http://127.22.144.130:45289/ using document root <none> and password file <none>
I20260811 04:47:22.425894 9020 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:22.425962 9020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:22.426143 9020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:22.428279 9020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "0b294d207c16b4ac54e4a47b69ac90e2"
server_key_iv: "6ffd4fa2d5b3bb86c1fb8110db619a6f"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.428838 9020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "0b294d207c16b4ac54e4a47b69ac90e2"
server_key_iv: "6ffd4fa2d5b3bb86c1fb8110db619a6f"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.432794 9020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:47:22.435452 9038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.436770 9020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:47:22.436913 9020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "0b294d207c16b4ac54e4a47b69ac90e2"
server_key_iv: "6ffd4fa2d5b3bb86c1fb8110db619a6f"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.437033 9020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:22.451021 9020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:22.454257 9020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:22.454500 9020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:22.455144 9020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:22.456001 9020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:22.456073 9020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.456146 9020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:22.456197 9020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.465215 9020 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:38455
I20260811 04:47:22.465238 9175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:38455 every 8 connection(s)
I20260811 04:47:22.466217 9020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:22.474931 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 9020
I20260811 04:47:22.475093 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:47:22.475409 23106 external_mini_cluster.cc:1428] Setting key 2103670a563c9e867ece8e514386bac8
Aug 11 04:47:22 dist-test-slave-ss4n krb5kdc[8729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:22.481153 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.468078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:44883 (local address 127.22.144.190:39763)
0811 04:47:22.468229 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.468232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.469122 (+ 890us) server_negotiation.cc:407] Connection header received
0811 04:47:22.470592 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.470596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.470670 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.470749 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:22.471964 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.472544 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.473817 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.474064 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.474181 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.477043 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:22.477046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:22.477048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:22.477072 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:22.478610 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.479326 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.479332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.479334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.479434 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.479788 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.479792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.479794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.480048 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:22.480156 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.480836 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.480973 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:47:22.483093 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.468447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39763 (local address 127.22.144.130:44883)
0811 04:47:22.468971 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.469014 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.470368 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.470927 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.470934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.471297 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:22.471807 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.471822 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.472691 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.472700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.473690 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.473701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.474238 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.474242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.474408 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.475087 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:22.475120 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:22.476871 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:22.478764 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:22.478774 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:22.478793 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:22.479195 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:22.479533 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:22.479535 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:22.479536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:22.479688 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:22.480157 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:22.480163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:22.480481 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.481452 (+ 971us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.481885 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":144,"threads_started":1}
I20260811 04:47:22.484573 9176 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39763
I20260811 04:47:22.484862 9176 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:22.485419 9176 heartbeater.cc:511] Master 127.22.144.190:39763 requested a full tablet report, sending...
I20260811 04:47:22.486608 8784 ts_manager.cc:194] Registered new tserver with Master: 25a1f09fd38a47d8a3a0ca0f3ad3c871 (127.22.144.130:38455)
I20260811 04:47:22.487277 8784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:44883
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:47:22.508993 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39763
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:22.623430 9183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:22.623735 9183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:22.623818 9183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:22.628528 9183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:22.628628 9183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:22.628715 9183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:22.634652 9183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:22.635776 9183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:22.636875 9183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:22.647064 9192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:22.647123 9188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:22.647125 9189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:22.647323 9183 server_base.cc:1034] running on GCE node
I20260811 04:47:22.647737 9183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:22.648415 9183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:22.649614 9183 hybrid_clock.cc:648] HybridClock initialized: now 1786423642649606 us; error 39 us; skew 500 ppm
Aug 11 04:47:22 dist-test-slave-ss4n krb5kdc[8729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:22.652657 9183 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:47:22.653671 9183 webserver.cc:460] Webserver started at http://127.22.144.131:40565/ using document root <none> and password file <none>
I20260811 04:47:22.654284 9183 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:22.654381 9183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:22.654621 9183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:22.656833 9183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "aa99e8a47c8b4b09831196d7c35a75ae"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "568ec8c344ced50872cad3a8df82d58f"
server_key_iv: "0c9ec7cd3d53ccb62d54aa78ebb452ba"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.657424 9183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "aa99e8a47c8b4b09831196d7c35a75ae"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "568ec8c344ced50872cad3a8df82d58f"
server_key_iv: "0c9ec7cd3d53ccb62d54aa78ebb452ba"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.661243 9183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 04:47:22.663802 9200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.665019 9183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:47:22.665174 9183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "aa99e8a47c8b4b09831196d7c35a75ae"
format_stamp: "Formatted at 2026-08-11 04:47:22 on dist-test-slave-ss4n"
server_key: "568ec8c344ced50872cad3a8df82d58f"
server_key_iv: "0c9ec7cd3d53ccb62d54aa78ebb452ba"
server_key_version: "encryptionkey@0"
I20260811 04:47:22.665303 9183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:22.707118 9183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:22.710923 9183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:22.711144 9183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:22.711822 9183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:22.712721 9183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:22.712795 9183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.712864 9183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:22.712915 9183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:22.722570 9183 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:34077
I20260811 04:47:22.722596 9334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:34077 every 8 connection(s)
I20260811 04:47:22.723559 9183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:22.727326 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 9183
I20260811 04:47:22.727474 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:47:22.727775 23106 external_mini_cluster.cc:1428] Setting key 7ca4e2e96ee4ff2258e0f982f5a8ffa5
Aug 11 04:47:22 dist-test-slave-ss4n krb5kdc[8729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423642, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:22.737465 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.725461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:38659 (local address 127.22.144.190:39763)
0811 04:47:22.725634 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.725639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.726519 (+ 880us) server_negotiation.cc:407] Connection header received
0811 04:47:22.728016 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.728021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.728081 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.728178 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:22.729658 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.730340 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.731451 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.731664 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.731742 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.734182 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:22.734185 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:22.734187 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:22.734211 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:22.735667 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.736113 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.736116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.736117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.736206 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.736497 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.736499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.736500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.736685 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:22.736759 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.737239 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.737342 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 04:47:22.738330 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.725887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39763 (local address 127.22.144.131:38659)
0811 04:47:22.726362 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.726407 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.727769 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.728392 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.728400 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.728839 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:22.729466 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.729528 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.730460 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.730463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.731345 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.731355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.731788 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.731790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.731939 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.732509 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:22.732525 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:22.734021 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:22.735765 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:22.735769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:22.735781 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:22.736017 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:22.736283 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:22.736285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:22.736286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:22.736412 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:22.736770 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:22.736774 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:22.737002 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.737171 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.737421 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260811 04:47:22.739511 9335 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39763
I20260811 04:47:22.739766 9335 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:22.740348 9335 heartbeater.cc:511] Master 127.22.144.190:39763 requested a full tablet report, sending...
I20260811 04:47:22.741434 8784 ts_manager.cc:194] Registered new tserver with Master: aa99e8a47c8b4b09831196d7c35a75ae (127.22.144.131:34077)
I20260811 04:47:22.742054 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:47:22.742208 8784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:38659
I20260811 04:47:22.751729 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.744245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.22.144.190:39763)
0811 04:47:22.744384 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.744389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.744587 (+ 198us) server_negotiation.cc:407] Connection header received
0811 04:47:22.744758 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.744762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.744804 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.744892 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:22.745655 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.746167 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.747029 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.747265 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.747370 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.748695 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:22.748699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:22.748702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:22.748727 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:22.749988 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.750420 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.750424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.750425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.750517 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:22.750823 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:22.750827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:22.750828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:22.751055 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:22.751210 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.751449 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.751567 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:47:22.754835 8784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:47:22.757948 8784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:47:22.773562 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.766597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:33015 (local address 127.0.0.1:33078)
0811 04:47:22.767378 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.767431 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.767561 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.768512 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.768517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.768542 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:22.768739 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.768746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.770372 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.770375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.771792 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.771803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.772956 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.772960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.773084 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.773127 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.773240 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.773363 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":56,"thread_start_us":79,"threads_started":1}
I20260811 04:47:22.773921 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.767486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.22.144.130:38455)
0811 04:47:22.767890 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.767902 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.768101 (+ 199us) server_negotiation.cc:407] Connection header received
0811 04:47:22.768203 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.768211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.768376 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.768536 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:22.769606 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.770729 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.772087 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.772885 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.772993 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.773106 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.773640 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.773762 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":149,"threads_started":1}
I20260811 04:47:22.774328 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.766764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38455 (local address 127.0.0.1:33972)
0811 04:47:22.767894 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.767908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.768003 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.768691 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.768693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.768705 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:22.768880 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.768888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.770880 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.770883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.771969 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.771977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.773182 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.773185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.773305 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.773330 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.773407 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.773515 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"mutex_wait_us":45,"thread_start_us":106,"threads_started":1}
I20260811 04:47:22.774437 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.766767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33078 (local address 127.22.144.129:33015)
0811 04:47:22.767984 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.767992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.768017 (+ 25us) server_negotiation.cc:407] Connection header received
0811 04:47:22.768099 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.768125 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.768339 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.768507 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:22.768887 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.770246 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.771923 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.772815 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.772963 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.773106 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.774139 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.774266 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1118,"thread_start_us":97,"threads_started":1}
I20260811 04:47:22.774655 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.767281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.22.144.131:34077)
0811 04:47:22.767782 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.767796 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.768150 (+ 354us) server_negotiation.cc:407] Connection header received
0811 04:47:22.768232 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.768237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.768378 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.768511 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:22.769011 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.770196 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.772750 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.773673 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.773801 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.773930 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.774399 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.774507 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":152,"threads_started":1}
I20260811 04:47:22.774509 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.766976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34077 (local address 127.0.0.1:42678)
0811 04:47:22.768039 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.768055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.768148 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.768665 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.768669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.768690 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:22.768884 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.768889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.770362 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.770365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.772618 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.772626 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.773998 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.774002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.774124 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.774149 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.774220 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.774293 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":69,"thread_start_us":86,"threads_started":1}
I20260811 04:47:22.777710 8934 tablet_service.cc:1467] Processing CreateTablet for tablet 9394cfe8c15340929519c959b244f964 (DEFAULT_TABLE table=test-table [id=df4569d7914a44eb8ed7056d151bdeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:22.777710 9096 tablet_service.cc:1467] Processing CreateTablet for tablet 9394cfe8c15340929519c959b244f964 (DEFAULT_TABLE table=test-table [id=df4569d7914a44eb8ed7056d151bdeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:22.778505 9255 tablet_service.cc:1467] Processing CreateTablet for tablet 9394cfe8c15340929519c959b244f964 (DEFAULT_TABLE table=test-table [id=df4569d7914a44eb8ed7056d151bdeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:22.778883 8934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9394cfe8c15340929519c959b244f964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:22.778883 9096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9394cfe8c15340929519c959b244f964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:22.779381 9255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9394cfe8c15340929519c959b244f964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:22.785993 9353 tablet_bootstrap.cc:492] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Bootstrap starting.
I20260811 04:47:22.787108 9354 tablet_bootstrap.cc:492] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4: Bootstrap starting.
I20260811 04:47:22.788933 9353 tablet_bootstrap.cc:654] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:22.789206 9354 tablet_bootstrap.cc:654] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:22.789856 9355 tablet_bootstrap.cc:492] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Bootstrap starting.
I20260811 04:47:22.790158 9354 log.cc:824] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:22.790208 9353 log.cc:824] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:22.791891 9355 tablet_bootstrap.cc:654] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:22.792394 9354 tablet_bootstrap.cc:492] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4: No bootstrap required, opened a new log
I20260811 04:47:22.792394 9353 tablet_bootstrap.cc:492] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: No bootstrap required, opened a new log
I20260811 04:47:22.792600 9354 ts_tablet_manager.cc:1397] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 04:47:22.792600 9353 ts_tablet_manager.cc:1397] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 04:47:22.793123 9355 log.cc:824] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:22.794878 9355 tablet_bootstrap.cc:492] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: No bootstrap required, opened a new log
I20260811 04:47:22.795095 9355 ts_tablet_manager.cc:1397] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 04:47:22.797137 9353 raft_consensus.cc:348] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.797168 9354 raft_consensus.cc:348] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.797461 9354 raft_consensus.cc:374] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:22.797461 9353 raft_consensus.cc:374] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:22.797564 9353 raft_consensus.cc:729] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa99e8a47c8b4b09831196d7c35a75ae, State: Initialized, Role: FOLLOWER
I20260811 04:47:22.797585 9354 raft_consensus.cc:729] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0baa556568404ad5aecec132807c09c4, State: Initialized, Role: FOLLOWER
I20260811 04:47:22.798105 9354 consensus_queue.cc:260] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.798105 9353 consensus_queue.cc:260] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.798410 9355 raft_consensus.cc:348] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.798740 9355 raft_consensus.cc:374] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:22.798843 9355 raft_consensus.cc:729] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a1f09fd38a47d8a3a0ca0f3ad3c871, State: Initialized, Role: FOLLOWER
I20260811 04:47:22.798851 9015 heartbeater.cc:503] Master 127.22.144.190:39763 was elected leader, sending a full tablet report...
I20260811 04:47:22.799093 9335 heartbeater.cc:503] Master 127.22.144.190:39763 was elected leader, sending a full tablet report...
I20260811 04:47:22.799172 9354 ts_tablet_manager.cc:1428] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260811 04:47:22.800251 9353 ts_tablet_manager.cc:1428] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260811 04:47:22.800345 9355 consensus_queue.cc:260] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.801074 9176 heartbeater.cc:503] Master 127.22.144.190:39763 was elected leader, sending a full tablet report...
I20260811 04:47:22.801317 9355 ts_tablet_manager.cc:1428] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:47:22.847886 9362 raft_consensus.cc:482] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:22.848093 9362 raft_consensus.cc:504] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.849226 9362 leader_election.cc:288] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0baa556568404ad5aecec132807c09c4 (127.22.144.129:33015), aa99e8a47c8b4b09831196d7c35a75ae (127.22.144.131:34077)
I20260811 04:47:22.853363 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.849634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:43453 (local address 127.22.144.131:34077)
0811 04:47:22.849781 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.849788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.849929 (+ 141us) server_negotiation.cc:407] Connection header received
0811 04:47:22.850111 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.850114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.850162 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.850258 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:22.850534 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.851204 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.852361 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.852771 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.852900 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.852954 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.853158 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.853233 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:47:22.853363 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.849524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:34077 (local address 127.22.144.130:43453)
0811 04:47:22.849788 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.849805 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.849924 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.850269 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.850272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.850293 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:22.850416 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.850422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.851315 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.851319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.852229 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.852239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.852900 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.852904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.853031 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.853049 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.853140 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.853226 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":1}
I20260811 04:47:22.853456 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.849817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:45733 (local address 127.22.144.129:33015)
0811 04:47:22.849943 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.849947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.850127 (+ 180us) server_negotiation.cc:407] Connection header received
0811 04:47:22.850289 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.850292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.850346 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.850433 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:22.850746 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.851342 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.852462 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.852879 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.853010 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.853060 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.853253 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.853329 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 04:47:22.853499 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.849591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:33015 (local address 127.22.144.130:45733)
0811 04:47:22.850027 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.850038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:22.850171 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:22.850445 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:22.850448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:22.850465 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:22.850594 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.850599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.851458 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.851464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.852328 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:22.852338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.853005 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.853008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:22.853110 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.853140 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 04:47:22.853233 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 04:47:22.853304 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":168,"threads_started":1}
I20260811 04:47:22.854315 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9394cfe8c15340929519c959b244f964" candidate_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" is_pre_election: true
I20260811 04:47:22.854451 8960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9394cfe8c15340929519c959b244f964" candidate_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0baa556568404ad5aecec132807c09c4" is_pre_election: true
I20260811 04:47:22.854668 9283 raft_consensus.cc:2454] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a1f09fd38a47d8a3a0ca0f3ad3c871 in term 0.
I20260811 04:47:22.854689 8960 raft_consensus.cc:2454] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a1f09fd38a47d8a3a0ca0f3ad3c871 in term 0.
I20260811 04:47:22.855204 9039 leader_election.cc:302] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0baa556568404ad5aecec132807c09c4, 25a1f09fd38a47d8a3a0ca0f3ad3c871, aa99e8a47c8b4b09831196d7c35a75ae; no voters:
I20260811 04:47:22.855564 9362 raft_consensus.cc:2781] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:47:22.855682 9362 raft_consensus.cc:482] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:47:22.855746 9362 raft_consensus.cc:3037] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:22.856850 9362 raft_consensus.cc:504] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.857252 9362 leader_election.cc:288] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [CANDIDATE]: Term 1 election: Requested vote from peers 0baa556568404ad5aecec132807c09c4 (127.22.144.129:33015), aa99e8a47c8b4b09831196d7c35a75ae (127.22.144.131:34077)
I20260811 04:47:22.857733 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9394cfe8c15340929519c959b244f964" candidate_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa99e8a47c8b4b09831196d7c35a75ae"
I20260811 04:47:22.857762 8960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9394cfe8c15340929519c959b244f964" candidate_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0baa556568404ad5aecec132807c09c4"
I20260811 04:47:22.857853 9283 raft_consensus.cc:3037] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:22.857915 8960 raft_consensus.cc:3037] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:22.859033 9283 raft_consensus.cc:2454] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a1f09fd38a47d8a3a0ca0f3ad3c871 in term 1.
I20260811 04:47:22.859205 8960 raft_consensus.cc:2454] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a1f09fd38a47d8a3a0ca0f3ad3c871 in term 1.
I20260811 04:47:22.859421 9040 leader_election.cc:302] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a1f09fd38a47d8a3a0ca0f3ad3c871, aa99e8a47c8b4b09831196d7c35a75ae; no voters:
I20260811 04:47:22.859619 9362 raft_consensus.cc:2781] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:22.859876 9362 raft_consensus.cc:686] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 1 LEADER]: Becoming Leader. State: Replica: 25a1f09fd38a47d8a3a0ca0f3ad3c871, State: Running, Role: LEADER
I20260811 04:47:22.860224 9362 consensus_queue.cc:237] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } }
I20260811 04:47:22.863958 8784 catalog_manager.cc:5495] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25a1f09fd38a47d8a3a0ca0f3ad3c871 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a1f09fd38a47d8a3a0ca0f3ad3c871" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 } health_report { overall_health: UNKNOWN } } }
I20260811 04:47:22.896344 9352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.892637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.22.144.130:38455)
0811 04:47:22.892785 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:22.892790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:22.892805 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:47:22.892911 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:22.892913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:22.892973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:22.893093 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:47:22.893361 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.893900 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.894783 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:22.895009 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:22.895123 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:22.895458 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:47:22.895896 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:47:22.896037 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:22.896102 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 04:47:22.896169 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:47:22.902566 8960 raft_consensus.cc:1261] T 9394cfe8c15340929519c959b244f964 P 0baa556568404ad5aecec132807c09c4 [term 1 FOLLOWER]: Refusing update from remote peer 25a1f09fd38a47d8a3a0ca0f3ad3c871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:22.902565 9283 raft_consensus.cc:1261] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 1 FOLLOWER]: Refusing update from remote peer 25a1f09fd38a47d8a3a0ca0f3ad3c871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:22.903334 9362 consensus_queue.cc:1035] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0baa556568404ad5aecec132807c09c4" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:22.903540 9364 consensus_queue.cc:1035] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa99e8a47c8b4b09831196d7c35a75ae" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 34077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:22.910657 9370 mvcc.cc:205] Tried to move back new op lower bound from 7317191241321095168 to 7317191241158463488. Current Snapshot: MvccSnapshot[applied={T|T < 7317191241321095168}]
I20260811 04:47:22.913077 9372 mvcc.cc:205] Tried to move back new op lower bound from 7317191241321095168 to 7317191241158463488. Current Snapshot: MvccSnapshot[applied={T|T < 7317191241321095168}]
I20260811 04:47:22.913287 9371 mvcc.cc:205] Tried to move back new op lower bound from 7317191241321095168 to 7317191241158463488. Current Snapshot: MvccSnapshot[applied={T|T < 7317191241321095168}]
I20260811 04:47:22.921401 8780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:22.921643 8780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:22.924403 8780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae96bbe0d50a4df0a6ec3e2b2a6fb5b9: Sending DeleteTablet for 3 replicas of tablet 9394cfe8c15340929519c959b244f964
I20260811 04:47:22.925370 8934 tablet_service.cc:1514] Processing DeleteTablet for tablet 9394cfe8c15340929519c959b244f964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:33078
I20260811 04:47:22.925519 9255 tablet_service.cc:1514] Processing DeleteTablet for tablet 9394cfe8c15340929519c959b244f964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:42678
I20260811 04:47:22.926431 9096 tablet_service.cc:1514] Processing DeleteTablet for tablet 9394cfe8c15340929519c959b244f964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:22 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:33972
I20260811 04:47:22.927016 9381 tablet_replica.cc:330] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: stopping tablet replica
I20260811 04:47:22.927109 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 8856
I20260811 04:47:22.928252 9381 raft_consensus.cc:2229] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:22.928707 9381 raft_consensus.cc:2258] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:22.932039 9382 tablet_replica.cc:330] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: stopping tablet replica
I20260811 04:47:22.932684 9382 raft_consensus.cc:2229] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:47:22.933228 9382 raft_consensus.cc:2258] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:22.935531 9381 ts_tablet_manager.cc:1905] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:22.937981 9382 ts_tablet_manager.cc:1905] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:22.939586 9381 ts_tablet_manager.cc:1918] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:22.939800 9381 log.cc:1196] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/9394cfe8c15340929519c959b244f964
I20260811 04:47:22.940317 9381 ts_tablet_manager.cc:1939] T 9394cfe8c15340929519c959b244f964 P aa99e8a47c8b4b09831196d7c35a75ae: Deleting consensus metadata
W20260811 04:47:22.940877 8764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:33015 (error 108)
I20260811 04:47:22.941033 9382 ts_tablet_manager.cc:1918] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:22.941129 9382 log.cc:1196] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/9394cfe8c15340929519c959b244f964
I20260811 04:47:22.941422 9382 ts_tablet_manager.cc:1939] T 9394cfe8c15340929519c959b244f964 P 25a1f09fd38a47d8a3a0ca0f3ad3c871: Deleting consensus metadata
I20260811 04:47:22.942072 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 9020
I20260811 04:47:22.942028 8765 catalog_manager.cc:4841] TS aa99e8a47c8b4b09831196d7c35a75ae (127.22.144.131:34077): tablet 9394cfe8c15340929519c959b244f964 (table test-table [id=df4569d7914a44eb8ed7056d151bdeee]) successfully deleted
I20260811 04:47:22.945843 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.945552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:33015 (local address 127.0.0.1:33084)
0811 04:47:22.945688 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.945766 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:33015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 04:47:22.946023 8764 catalog_manager.cc:4584] TS 0baa556568404ad5aecec132807c09c4 (127.22.144.129:33015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9394cfe8c15340929519c959b244f964: Network error: Client connection negotiation failed: client connection to 127.22.144.129:33015: connect: Connection refused (error 111)
I20260811 04:47:22.951884 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 9183
I20260811 04:47:22.954901 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:22.952007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38455 (local address 127.0.0.1:33994)
0811 04:47:22.954683 (+ 2676us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:22.954777 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.130:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2542}
I20260811 04:47:22.960934 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 8747
2026-08-11T04:47:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2502 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T04:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:22Z Disabled control of system clock
I20260811 04:47:22.997820 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41899
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:41899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 04:47:23.115020 9389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:23.115314 9389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:23.115350 9389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:23.119444 9389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:23.119521 9389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:23.119537 9389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:23.119554 9389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:23.119570 9389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:23.124560 9389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:41899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:23.125837 9389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:23.127264 9389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:23.136185 9395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:23.136185 9397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:23.136194 9394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:23.136487 9389 server_base.cc:1034] running on GCE node
I20260811 04:47:23.137140 9389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:23.138451 9389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:23.139624 9389 hybrid_clock.cc:648] HybridClock initialized: now 1786423643139591 us; error 51 us; skew 500 ppm
I20260811 04:47:23.141731 9389 webserver.cc:460] Webserver started at http://127.22.144.190:34189/ using document root <none> and password file <none>
I20260811 04:47:23.142311 9389 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:23.142376 9389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:23.142551 9389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:23.144666 9389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "5de5c3d80d63361270a49eb47924ff1d"
server_key_iv: "cac394e163731308bd0c8d9d72f7ded4"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.145223 9389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "5de5c3d80d63361270a49eb47924ff1d"
server_key_iv: "cac394e163731308bd0c8d9d72f7ded4"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.149127 9389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 04:47:23.151762 9404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.153105 9389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 04:47:23.153223 9389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "5de5c3d80d63361270a49eb47924ff1d"
server_key_iv: "cac394e163731308bd0c8d9d72f7ded4"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.153321 9389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:23.164932 9389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:23.165704 9389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:23.165867 9389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:23.173465 9389 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:41899
I20260811 04:47:23.173621 9467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:41899 every 8 connection(s)
I20260811 04:47:23.174575 9389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:23.177839 9468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:23.183560 9468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f: Bootstrap starting.
I20260811 04:47:23.184451 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 9389
I20260811 04:47:23.184587 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:23.184931 23106 external_mini_cluster.cc:1428] Setting key 77cfe9f227491c385a8eb49e530ed537
I20260811 04:47:23.186733 9468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:23.187498 9468 log.cc:824] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:23.189800 9468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f: No bootstrap required, opened a new log
I20260811 04:47:23.194281 9468 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41899 } }
I20260811 04:47:23.194527 9468 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:23.194563 9468 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf58b54b6a44fb7b4c4155e6e61d60f, State: Initialized, Role: FOLLOWER
I20260811 04:47:23.194896 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.186238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48278 (local address 127.22.144.190:41899)
0811 04:47:23.186954 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.186970 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.187007 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:47:23.187747 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.187766 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.188209 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.188536 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:23.188885 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.190025 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.191243 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.191769 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.191891 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.192906 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:23.192911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:23.192926 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:23.193341 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:23.193399 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:23.193420 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:23.193618 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.193705 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.194016 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":157,"threads_started":1}
I20260811 04:47:23.195097 9468 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41899 } }
I20260811 04:47:23.195273 9468 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:23.195343 9468 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:23.195446 9468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:23.196502 9468 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41899 } }
I20260811 04:47:23.196854 9468 leader_election.cc:302] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecf58b54b6a44fb7b4c4155e6e61d60f; no voters:
I20260811 04:47:23.197199 9468 leader_election.cc:288] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:23.197316 9473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:23.197634 9473 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [term 1 LEADER]: Becoming Leader. State: Replica: ecf58b54b6a44fb7b4c4155e6e61d60f, State: Running, Role: LEADER
I20260811 04:47:23.197956 9473 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41899 } }
I20260811 04:47:23.198562 9468 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:23.199611 9474 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41899 } } }
I20260811 04:47:23.199744 9474 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:23.199872 9475 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecf58b54b6a44fb7b4c4155e6e61d60f. Latest consensus state: current_term: 1 leader_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf58b54b6a44fb7b4c4155e6e61d60f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41899 } } }
I20260811 04:47:23.200042 9475 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:23.200419 9483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:23.203709 9483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:23.210237 9483 catalog_manager.cc:1347] Generated new cluster ID: cef409173bdb4b828ccf50d452ad571f
I20260811 04:47:23.210335 9483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:23.224747 9483 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:23.225848 9483 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:23.230636 9483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f: Generated new TSK 0
I20260811 04:47:23.231616 9483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:23.241531 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41899
--builtin_ntp_servers=127.22.144.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 04:47:23.361351 9496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:23.361680 9496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:23.361724 9496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:23.365887 9496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:23.365953 9496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:23.366009 9496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:23.370862 9496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:23.372020 9496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:23.373466 9496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:23.383448 9502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:23.383536 9496 server_base.cc:1034] running on GCE node
W20260811 04:47:23.383502 9503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:23.383455 9505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:23.384148 9496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:23.384856 9496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:23.386030 9496 hybrid_clock.cc:648] HybridClock initialized: now 1786423643386009 us; error 30 us; skew 500 ppm
I20260811 04:47:23.388154 9496 webserver.cc:460] Webserver started at http://127.22.144.129:42137/ using document root <none> and password file <none>
I20260811 04:47:23.388795 9496 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:23.388867 9496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:23.389091 9496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:23.391373 9496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "d4b9ec9112ab4d5095471c282c432bc7"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "34b330bb4ca7d683a099a1d235d95ca4"
server_key_iv: "3d42ba7a04edcfc2142f47fd93cbcf15"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.392084 9496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "d4b9ec9112ab4d5095471c282c432bc7"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "34b330bb4ca7d683a099a1d235d95ca4"
server_key_iv: "3d42ba7a04edcfc2142f47fd93cbcf15"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.396325 9496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:47:23.399191 9513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.400589 9496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:47:23.400722 9496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "d4b9ec9112ab4d5095471c282c432bc7"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "34b330bb4ca7d683a099a1d235d95ca4"
server_key_iv: "3d42ba7a04edcfc2142f47fd93cbcf15"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.400844 9496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:23.423466 9496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:23.424249 9496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:23.424467 9496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:23.425132 9496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:23.426250 9496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:23.426326 9496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.426404 9496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:23.426448 9496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.435978 9496 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:33579
I20260811 04:47:23.436014 9650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:33579 every 8 connection(s)
I20260811 04:47:23.436952 9496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:47:23.440639 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 9496
I20260811 04:47:23.440757 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:47:23.441057 23106 external_mini_cluster.cc:1428] Setting key 1e991a91668dfca98ab38bf81ff3768e
I20260811 04:47:23.443445 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41899
--builtin_ntp_servers=127.22.144.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 04:47:23.447547 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.439051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:34277 (local address 127.22.144.190:41899)
0811 04:47:23.439197 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.439201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.440393 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 04:47:23.442069 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.442073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.442118 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.442207 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:23.443232 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.444040 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.445304 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.445622 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.445738 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.447035 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:23.447039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:23.447041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:23.447054 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:23.447076 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:23.447084 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:23.447152 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.447371 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.447430 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:47:23.448930 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.439487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41899 (local address 127.22.144.129:34277)
0811 04:47:23.440208 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.440256 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.441812 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.442419 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.442428 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.442549 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:23.443072 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.443087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.444187 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.444190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.445159 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.445170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.445803 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.445808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.445983 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.446686 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:23.446690 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:23.446756 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:23.446853 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:23.447187 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:23.447201 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:23.447210 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.447359 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.447698 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":130,"threads_started":1}
I20260811 04:47:23.450376 9651 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41899
I20260811 04:47:23.450668 9651 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:23.451452 9651 heartbeater.cc:511] Master 127.22.144.190:41899 requested a full tablet report, sending...
I20260811 04:47:23.453680 9425 ts_manager.cc:194] Registered new tserver with Master: d4b9ec9112ab4d5095471c282c432bc7 (127.22.144.129:33579)
I20260811 04:47:23.455722 9425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.129:34277
W20260811 04:47:23.567098 9654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:23.567401 9654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:23.567442 9654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:23.571689 9654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:23.571750 9654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:23.571805 9654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:23.576546 9654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:23.577741 9654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:23.579207 9654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:23.589781 9659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:23.589881 9654 server_base.cc:1034] running on GCE node
W20260811 04:47:23.589792 9663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:23.589807 9660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:23.590548 9654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:23.591207 9654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:23.592404 9654 hybrid_clock.cc:648] HybridClock initialized: now 1786423643592387 us; error 30 us; skew 500 ppm
I20260811 04:47:23.594581 9654 webserver.cc:460] Webserver started at http://127.22.144.130:41797/ using document root <none> and password file <none>
I20260811 04:47:23.595237 9654 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:23.595314 9654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:23.595669 9654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:23.598091 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "5ae3d14095984bd6878d90be30af96dd"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "d74f89cdefcb05fc89b6be07288b2e53"
server_key_iv: "a711d71343657f5d6695aa73f25289f1"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.598742 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "5ae3d14095984bd6878d90be30af96dd"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "d74f89cdefcb05fc89b6be07288b2e53"
server_key_iv: "a711d71343657f5d6695aa73f25289f1"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.603134 9654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:23.606184 9669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.607636 9654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 04:47:23.607813 9654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "5ae3d14095984bd6878d90be30af96dd"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "d74f89cdefcb05fc89b6be07288b2e53"
server_key_iv: "a711d71343657f5d6695aa73f25289f1"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.607946 9654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:23.624689 9654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:23.625573 9654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:23.625789 9654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:23.626441 9654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:23.627461 9654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:23.627539 9654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.627614 9654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:23.627660 9654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.637462 9654 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:43157
I20260811 04:47:23.637562 9796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:43157 every 8 connection(s)
I20260811 04:47:23.638564 9654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:23.642277 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 9654
I20260811 04:47:23.642427 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:47:23.642763 23106 external_mini_cluster.cc:1428] Setting key fd65a3e7c5e12fd6a39c942d02a10479
I20260811 04:47:23.645108 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41899
--builtin_ntp_servers=127.22.144.148:35401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 04:47:23.648973 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.640662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:43535 (local address 127.22.144.190:41899)
0811 04:47:23.640854 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.640860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.642020 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 04:47:23.643645 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.643648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.643724 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.643910 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:23.644868 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.645505 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.646723 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.647104 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.647231 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.648481 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:23.648484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:23.648486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:23.648501 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:23.648524 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:23.648531 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:23.648597 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.648796 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.648855 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 04:47:23.650353 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.641047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41899 (local address 127.22.144.130:43535)
0811 04:47:23.641841 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.641887 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.643414 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.644107 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.644114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.644206 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:23.644721 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.644734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.645676 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.645679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.646584 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.646594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.647277 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.647281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.647435 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.648148 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:23.648153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:23.648212 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:23.648304 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:23.648625 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:23.648638 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:23.648647 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.648786 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.649165 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260811 04:47:23.651648 9797 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41899
I20260811 04:47:23.651904 9797 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:23.652452 9797 heartbeater.cc:511] Master 127.22.144.190:41899 requested a full tablet report, sending...
I20260811 04:47:23.653719 9425 ts_manager.cc:194] Registered new tserver with Master: 5ae3d14095984bd6878d90be30af96dd (127.22.144.130:43157)
I20260811 04:47:23.654327 9425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.130:43535
W20260811 04:47:23.764183 9801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:23.764467 9801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:23.764505 9801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:23.768817 9801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:23.768900 9801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:23.768957 9801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:23.773871 9801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:23.775051 9801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:23.776504 9801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:23.786473 9809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:23.786541 9807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:23.786489 9806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:23.786844 9801 server_base.cc:1034] running on GCE node
I20260811 04:47:23.787324 9801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:23.788013 9801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:23.789213 9801 hybrid_clock.cc:648] HybridClock initialized: now 1786423643789201 us; error 32 us; skew 500 ppm
I20260811 04:47:23.791299 9801 webserver.cc:460] Webserver started at http://127.22.144.131:43323/ using document root <none> and password file <none>
I20260811 04:47:23.791945 9801 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:23.792049 9801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:23.792304 9801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:23.794584 9801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "253f7a2ac6814beeb6f3090e13713ec5"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "51e70c0a659b6e9e60ea309687b7a998"
server_key_iv: "81d8e90a65313b07f94e121a7681c31d"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.795202 9801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "253f7a2ac6814beeb6f3090e13713ec5"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "51e70c0a659b6e9e60ea309687b7a998"
server_key_iv: "81d8e90a65313b07f94e121a7681c31d"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.799357 9801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 04:47:23.802196 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.803650 9801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:23.803828 9801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "253f7a2ac6814beeb6f3090e13713ec5"
format_stamp: "Formatted at 2026-08-11 04:47:23 on dist-test-slave-ss4n"
server_key: "51e70c0a659b6e9e60ea309687b7a998"
server_key_iv: "81d8e90a65313b07f94e121a7681c31d"
server_key_version: "encryptionkey@0"
I20260811 04:47:23.803934 9801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:23.817137 9801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:23.817922 9801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:23.818130 9801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:23.818758 9801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:23.819746 9801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:23.819793 9801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.819865 9801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:23.819902 9801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:23.829248 9801 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:45217
I20260811 04:47:23.829339 9957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:45217 every 8 connection(s)
I20260811 04:47:23.830310 9801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:23.833514 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 9801
I20260811 04:47:23.833649 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:47:23.833954 23106 external_mini_cluster.cc:1428] Setting key 7bcd26204fb144b44ac01abcad9d83b2
I20260811 04:47:23.840828 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.832316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:43805 (local address 127.22.144.190:41899)
0811 04:47:23.832483 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.832487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.833719 (+ 1232us) server_negotiation.cc:407] Connection header received
0811 04:47:23.835630 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.835633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.835687 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.835758 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:23.837021 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.837702 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.838900 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.839111 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.839193 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.840392 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:23.840395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:23.840397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:23.840408 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:23.840424 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:23.840431 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:23.840484 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.840626 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.840710 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 04:47:23.841754 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.832751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:41899 (local address 127.22.144.131:43805)
0811 04:47:23.833459 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.833546 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.835369 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.835976 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.835986 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.836103 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:23.836865 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.836881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.837828 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.837831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.838785 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.838796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.839249 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.839251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.839395 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.840143 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:23.840145 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:23.840188 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:23.840257 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:23.840504 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:23.840510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:23.840515 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.840658 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.840928 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":134,"threads_started":1}
I20260811 04:47:23.842965 9958 heartbeater.cc:347] Connected to a master server at 127.22.144.190:41899
I20260811 04:47:23.843230 9958 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:23.843787 9958 heartbeater.cc:511] Master 127.22.144.190:41899 requested a full tablet report, sending...
I20260811 04:47:23.844874 9425 ts_manager.cc:194] Registered new tserver with Master: 253f7a2ac6814beeb6f3090e13713ec5 (127.22.144.131:45217)
I20260811 04:47:23.845734 9425 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.131:43805
I20260811 04:47:23.848374 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:47:23.853932 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.850385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.22.144.190:41899)
0811 04:47:23.850517 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.850521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.850662 (+ 141us) server_negotiation.cc:407] Connection header received
0811 04:47:23.850766 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.850768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.850804 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.850873 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:23.851095 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.851610 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.852456 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.852659 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.852734 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.853583 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:23.853586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:23.853588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:23.853600 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:23.853616 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:23.853623 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:23.853680 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.853773 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.853822 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 04:47:23.856987 9425 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:47:23.860088 9425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:47:23.875193 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.869157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:43157 (local address 127.0.0.1:50096)
0811 04:47:23.869780 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.869794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.869880 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.870894 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.870900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.870931 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:23.871025 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.871031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.872435 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.872439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.873447 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.873457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.874617 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.874621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.874741 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.874786 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.874904 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.875018 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":17,"thread_start_us":118,"threads_started":1}
I20260811 04:47:23.875247 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.869895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.22.144.130:43157)
0811 04:47:23.870405 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.870414 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.870433 (+ 19us) server_negotiation.cc:407] Connection header received
0811 04:47:23.870530 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.870535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.870742 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.870892 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:23.871158 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.872297 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.873676 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.874482 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.874608 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.874796 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.874949 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.875052 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":147,"threads_started":1}
I20260811 04:47:23.875392 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.869665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:45217 (local address 127.0.0.1:36424)
0811 04:47:23.870066 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.870078 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.870157 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.871090 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.871093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.871113 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:23.871202 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.871209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.872257 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.872260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.873594 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.873607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.874909 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.874914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.875014 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.875037 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.875121 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.875195 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":92,"thread_start_us":99,"threads_started":1}
I20260811 04:47:23.875804 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.870308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.22.144.131:45217)
0811 04:47:23.870670 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.870680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.870701 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:47:23.870773 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.870778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.870960 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.871108 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:23.871324 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.872090 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.873737 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.874482 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.874608 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.874796 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.875522 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.875626 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":147,"threads_started":1}
I20260811 04:47:23.876144 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.868807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:33579 (local address 127.0.0.1:37660)
0811 04:47:23.869234 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.869293 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.869426 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.870671 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.870678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.870701 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:23.870862 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.870872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.872435 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.872439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.874334 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.874344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.875738 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.875742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.875841 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.875864 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.875936 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.876018 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260811 04:47:23.876575 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.869003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37660 (local address 127.22.144.129:33579)
0811 04:47:23.870065 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.870078 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.870106 (+ 28us) server_negotiation.cc:407] Connection header received
0811 04:47:23.870189 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.870196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.870429 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.870595 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:23.871014 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.872297 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.874480 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.875323 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.875465 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.875669 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.876321 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.876421 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":195,"threads_started":1}
I20260811 04:47:23.878692 9724 tablet_service.cc:1467] Processing CreateTablet for tablet 24dd1cd66b0849c98662a9127011af2c (DEFAULT_TABLE table=test-table [id=e45fe8522d164e6caec7d8c593a7d59d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:23.879117 9568 tablet_service.cc:1467] Processing CreateTablet for tablet 24dd1cd66b0849c98662a9127011af2c (DEFAULT_TABLE table=test-table [id=e45fe8522d164e6caec7d8c593a7d59d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:23.879037 9880 tablet_service.cc:1467] Processing CreateTablet for tablet 24dd1cd66b0849c98662a9127011af2c (DEFAULT_TABLE table=test-table [id=e45fe8522d164e6caec7d8c593a7d59d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:23.879884 9724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24dd1cd66b0849c98662a9127011af2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:23.880074 9880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24dd1cd66b0849c98662a9127011af2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:23.880074 9568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24dd1cd66b0849c98662a9127011af2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:23.887030 9979 tablet_bootstrap.cc:492] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Bootstrap starting.
I20260811 04:47:23.887660 9980 tablet_bootstrap.cc:492] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: Bootstrap starting.
I20260811 04:47:23.888041 9981 tablet_bootstrap.cc:492] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Bootstrap starting.
I20260811 04:47:23.889559 9979 tablet_bootstrap.cc:654] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:23.889894 9981 tablet_bootstrap.cc:654] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:23.889966 9980 tablet_bootstrap.cc:654] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:23.890503 9979 log.cc:824] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:23.890619 9981 log.cc:824] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:23.890820 9980 log.cc:824] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:23.892747 9979 tablet_bootstrap.cc:492] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: No bootstrap required, opened a new log
I20260811 04:47:23.892793 9981 tablet_bootstrap.cc:492] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: No bootstrap required, opened a new log
I20260811 04:47:23.892794 9980 tablet_bootstrap.cc:492] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: No bootstrap required, opened a new log
I20260811 04:47:23.892979 9981 ts_tablet_manager.cc:1397] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 04:47:23.892979 9980 ts_tablet_manager.cc:1397] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 04:47:23.892988 9979 ts_tablet_manager.cc:1397] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 04:47:23.896521 9979 raft_consensus.cc:348] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.896888 9979 raft_consensus.cc:374] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:23.896987 9979 raft_consensus.cc:729] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae3d14095984bd6878d90be30af96dd, State: Initialized, Role: FOLLOWER
I20260811 04:47:23.897367 9981 raft_consensus.cc:348] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.897368 9980 raft_consensus.cc:348] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.897665 9980 raft_consensus.cc:374] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:23.897571 9979 consensus_queue.cc:260] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.897746 9980 raft_consensus.cc:729] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b9ec9112ab4d5095471c282c432bc7, State: Initialized, Role: FOLLOWER
I20260811 04:47:23.897814 9981 raft_consensus.cc:374] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:23.897907 9981 raft_consensus.cc:729] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253f7a2ac6814beeb6f3090e13713ec5, State: Initialized, Role: FOLLOWER
I20260811 04:47:23.898329 9797 heartbeater.cc:503] Master 127.22.144.190:41899 was elected leader, sending a full tablet report...
I20260811 04:47:23.898326 9980 consensus_queue.cc:260] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.898447 9981 consensus_queue.cc:260] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.898790 9979 ts_tablet_manager.cc:1428] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 04:47:23.899442 9980 ts_tablet_manager.cc:1428] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:47:23.899494 9981 ts_tablet_manager.cc:1428] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 04:47:23.899922 9651 heartbeater.cc:503] Master 127.22.144.190:41899 was elected leader, sending a full tablet report...
I20260811 04:47:23.900914 9958 heartbeater.cc:503] Master 127.22.144.190:41899 was elected leader, sending a full tablet report...
W20260811 04:47:23.938329 9652 tablet.cc:2369] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:47:23.966250 9986 raft_consensus.cc:482] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:23.966529 9986 raft_consensus.cc:504] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.967717 9986 leader_election.cc:288] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4b9ec9112ab4d5095471c282c432bc7 (127.22.144.129:33579), 253f7a2ac6814beeb6f3090e13713ec5 (127.22.144.131:45217)
I20260811 04:47:23.971451 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.968136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:39109 (local address 127.22.144.129:33579)
0811 04:47:23.968309 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.968315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.968334 (+ 19us) server_negotiation.cc:407] Connection header received
0811 04:47:23.968400 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.968403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.968458 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.968559 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:23.968864 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.969373 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.970462 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.970876 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.971016 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.971071 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.971262 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.971337 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 04:47:23.971452 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.968011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:33579 (local address 127.22.144.130:39109)
0811 04:47:23.968181 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.968199 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.968299 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.968585 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.968588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.968606 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:23.968732 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.968739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.969520 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.969526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.970317 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.970326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.971009 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.971013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.971113 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.971137 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.971242 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.971314 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 04:47:23.972020 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.968325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:45217 (local address 127.22.144.130:51753)
0811 04:47:23.968793 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:23.968804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:23.968917 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:23.969166 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:23.969169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:23.969184 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:23.969315 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.969321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.970076 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.970081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.970861 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:23.970870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.971531 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.971535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:23.971654 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.971679 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 04:47:23.971805 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 04:47:23.971886 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":162,"threads_started":1}
I20260811 04:47:23.972041 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:23.968468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:51753 (local address 127.22.144.131:45217)
0811 04:47:23.968623 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:23.968630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:23.968888 (+ 258us) server_negotiation.cc:407] Connection header received
0811 04:47:23.969009 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:23.969014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:23.969061 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:23.969157 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:23.969426 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.969962 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.970992 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:23.971385 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:23.971522 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:23.971567 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:23.971856 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 04:47:23.971950 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 04:47:23.972165 9594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd1cd66b0849c98662a9127011af2c" candidate_uuid: "5ae3d14095984bd6878d90be30af96dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4b9ec9112ab4d5095471c282c432bc7" is_pre_election: true
I20260811 04:47:23.972527 9594 raft_consensus.cc:2454] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae3d14095984bd6878d90be30af96dd in term 0.
I20260811 04:47:23.972637 9902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd1cd66b0849c98662a9127011af2c" candidate_uuid: "5ae3d14095984bd6878d90be30af96dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "253f7a2ac6814beeb6f3090e13713ec5" is_pre_election: true
I20260811 04:47:23.972929 9902 raft_consensus.cc:2454] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae3d14095984bd6878d90be30af96dd in term 0.
I20260811 04:47:23.973064 9673 leader_election.cc:302] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae3d14095984bd6878d90be30af96dd, d4b9ec9112ab4d5095471c282c432bc7; no voters:
I20260811 04:47:23.973410 9986 raft_consensus.cc:2781] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:47:23.973532 9986 raft_consensus.cc:482] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:47:23.973592 9986 raft_consensus.cc:3037] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:23.974745 9986 raft_consensus.cc:504] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.975039 9986 leader_election.cc:288] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [CANDIDATE]: Term 1 election: Requested vote from peers d4b9ec9112ab4d5095471c282c432bc7 (127.22.144.129:33579), 253f7a2ac6814beeb6f3090e13713ec5 (127.22.144.131:45217)
I20260811 04:47:23.975467 9594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd1cd66b0849c98662a9127011af2c" candidate_uuid: "5ae3d14095984bd6878d90be30af96dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4b9ec9112ab4d5095471c282c432bc7"
I20260811 04:47:23.975503 9902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24dd1cd66b0849c98662a9127011af2c" candidate_uuid: "5ae3d14095984bd6878d90be30af96dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "253f7a2ac6814beeb6f3090e13713ec5"
I20260811 04:47:23.975605 9594 raft_consensus.cc:3037] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:23.975652 9902 raft_consensus.cc:3037] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:23.976662 9594 raft_consensus.cc:2454] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae3d14095984bd6878d90be30af96dd in term 1.
I20260811 04:47:23.976739 9902 raft_consensus.cc:2454] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae3d14095984bd6878d90be30af96dd in term 1.
I20260811 04:47:23.977001 9673 leader_election.cc:302] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae3d14095984bd6878d90be30af96dd, d4b9ec9112ab4d5095471c282c432bc7; no voters:
I20260811 04:47:23.977186 9986 raft_consensus.cc:2781] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:23.977438 9986 raft_consensus.cc:686] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 1 LEADER]: Becoming Leader. State: Replica: 5ae3d14095984bd6878d90be30af96dd, State: Running, Role: LEADER
I20260811 04:47:23.977872 9986 consensus_queue.cc:237] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } }
I20260811 04:47:23.981199 9423 catalog_manager.cc:5495] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae3d14095984bd6878d90be30af96dd (127.22.144.130). New cstate: current_term: 1 leader_uuid: "5ae3d14095984bd6878d90be30af96dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae3d14095984bd6878d90be30af96dd" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 43157 } health_report { overall_health: HEALTHY } } }
I20260811 04:47:24.007267 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:24.003133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.22.144.130:43157)
0811 04:47:24.003283 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:24.003286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:24.003301 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:47:24.003379 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:24.003381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:24.003438 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:24.003528 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:47:24.003808 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:24.004484 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:24.005729 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:24.005946 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:24.006029 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:24.006332 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:47:24.006771 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:47:24.006866 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:24.007038 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 04:47:24.007111 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 04:47:24.013731 9594 raft_consensus.cc:1261] T 24dd1cd66b0849c98662a9127011af2c P d4b9ec9112ab4d5095471c282c432bc7 [term 1 FOLLOWER]: Refusing update from remote peer 5ae3d14095984bd6878d90be30af96dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:24.013795 9902 raft_consensus.cc:1261] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 1 FOLLOWER]: Refusing update from remote peer 5ae3d14095984bd6878d90be30af96dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:24.014526 9991 consensus_queue.cc:1035] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4b9ec9112ab4d5095471c282c432bc7" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 33579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:24.014710 9986 consensus_queue.cc:1035] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "253f7a2ac6814beeb6f3090e13713ec5" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 45217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:24.033881 9423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:24.034065 9423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:24.037072 9423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecf58b54b6a44fb7b4c4155e6e61d60f: Sending DeleteTablet for 3 replicas of tablet 24dd1cd66b0849c98662a9127011af2c
I20260811 04:47:24.038267 9880 tablet_service.cc:1514] Processing DeleteTablet for tablet 24dd1cd66b0849c98662a9127011af2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:24 UTC) from {username='slave'} at 127.0.0.1:36424
I20260811 04:47:24.038352 9724 tablet_service.cc:1514] Processing DeleteTablet for tablet 24dd1cd66b0849c98662a9127011af2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:24 UTC) from {username='slave'} at 127.0.0.1:50096
I20260811 04:47:24.038944 9568 tablet_service.cc:1514] Processing DeleteTablet for tablet 24dd1cd66b0849c98662a9127011af2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:24 UTC) from {username='slave'} at 127.0.0.1:37660
I20260811 04:47:24.039134 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 9496
I20260811 04:47:24.041229 10007 tablet_replica.cc:330] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: stopping tablet replica
I20260811 04:47:24.041867 10007 raft_consensus.cc:2229] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:24.042330 10007 raft_consensus.cc:2258] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:24.044343 10007 ts_tablet_manager.cc:1905] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:24.045811 10006 tablet_replica.cc:330] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: stopping tablet replica
I20260811 04:47:24.046219 10006 raft_consensus.cc:2229] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:47:24.046785 10006 raft_consensus.cc:2258] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:24.048223 10007 ts_tablet_manager.cc:1918] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:24.048408 10007 log.cc:1196] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/24dd1cd66b0849c98662a9127011af2c
I20260811 04:47:24.048889 10007 ts_tablet_manager.cc:1939] T 24dd1cd66b0849c98662a9127011af2c P 253f7a2ac6814beeb6f3090e13713ec5: Deleting consensus metadata
I20260811 04:47:24.048949 10006 ts_tablet_manager.cc:1905] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:24.050544 9410 catalog_manager.cc:4841] TS 253f7a2ac6814beeb6f3090e13713ec5 (127.22.144.131:45217): tablet 24dd1cd66b0849c98662a9127011af2c (table test-table [id=e45fe8522d164e6caec7d8c593a7d59d]) successfully deleted
I20260811 04:47:24.051872 10006 ts_tablet_manager.cc:1918] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:24.052029 10006 log.cc:1196] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/24dd1cd66b0849c98662a9127011af2c
I20260811 04:47:24.052363 10006 ts_tablet_manager.cc:1939] T 24dd1cd66b0849c98662a9127011af2c P 5ae3d14095984bd6878d90be30af96dd: Deleting consensus metadata
I20260811 04:47:24.053653 9405 catalog_manager.cc:4841] TS 5ae3d14095984bd6878d90be30af96dd (127.22.144.130:43157): tablet 24dd1cd66b0849c98662a9127011af2c (table test-table [id=e45fe8522d164e6caec7d8c593a7d59d]) successfully deleted
I20260811 04:47:24.054592 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 9654
W20260811 04:47:24.054571 9410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:33579 (error 108)
I20260811 04:47:24.059891 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:24.059443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:33579 (local address 127.0.0.1:37676)
0811 04:47:24.059663 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:24.059768 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260811 04:47:24.060137 9410 catalog_manager.cc:4584] TS d4b9ec9112ab4d5095471c282c432bc7 (127.22.144.129:33579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24dd1cd66b0849c98662a9127011af2c: Network error: Client connection negotiation failed: client connection to 127.22.144.129:33579: connect: Connection refused (error 111)
I20260811 04:47:24.067301 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 9801
I20260811 04:47:24.075521 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 9389
2026-08-11T04:47:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1112 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:24 dist-test-slave-ss4n krb5kdc[10012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:24 dist-test-slave-ss4n krb5kdc[10012](info): set up 2 sockets
Aug 11 04:47:24 dist-test-slave-ss4n krb5kdc[10012](info): commencing operation
krb5kdc: starting...
W20260811 04:47:25.212775 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:25 dist-test-slave-ss4n krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:25.362144 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:36615
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:36615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:25.483732 10028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:25.484095 10028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:25.484186 10028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:25.488517 10028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:25.488624 10028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:25.488662 10028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:25.488713 10028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:25.488765 10028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:25.496073 10028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:36615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:36615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:25.497296 10028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:25.498479 10028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:25.508080 10036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:25.508270 10028 server_base.cc:1034] running on GCE node
W20260811 04:47:25.508127 10038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:25.508160 10035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:25.509044 10028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:25.510583 10028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:25.511781 10028 hybrid_clock.cc:648] HybridClock initialized: now 1786423645511764 us; error 34 us; skew 500 ppm
Aug 11 04:47:25 dist-test-slave-ss4n krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423645, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:25.514951 10028 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:25.515972 10028 webserver.cc:460] Webserver started at http://127.22.144.190:44687/ using document root <none> and password file <none>
I20260811 04:47:25.516541 10028 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:25.516604 10028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:25.516785 10028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:25.519208 10028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "3fae8f4ead874f7da0f1d03c84ce25a1"
format_stamp: "Formatted at 2026-08-11 04:47:25 on dist-test-slave-ss4n"
server_key: "a888c910975da1492e7a21a2f28c2770"
server_key_iv: "92b556332aea8eade283d0e856e07a3a"
server_key_version: "encryptionkey@0"
I20260811 04:47:25.519925 10028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "3fae8f4ead874f7da0f1d03c84ce25a1"
format_stamp: "Formatted at 2026-08-11 04:47:25 on dist-test-slave-ss4n"
server_key: "a888c910975da1492e7a21a2f28c2770"
server_key_iv: "92b556332aea8eade283d0e856e07a3a"
server_key_version: "encryptionkey@0"
I20260811 04:47:25.524435 10028 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 04:47:25.527750 10048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:25.529390 10028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:25.529666 10028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "3fae8f4ead874f7da0f1d03c84ce25a1"
format_stamp: "Formatted at 2026-08-11 04:47:25 on dist-test-slave-ss4n"
server_key: "a888c910975da1492e7a21a2f28c2770"
server_key_iv: "92b556332aea8eade283d0e856e07a3a"
server_key_version: "encryptionkey@0"
I20260811 04:47:25.529830 10028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:25.547226 10028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:25.551308 10028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:25.551553 10028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:25.560118 10028 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:36615
I20260811 04:47:25.560141 10105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:36615 every 8 connection(s)
I20260811 04:47:25.561419 10028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:25.564579 10106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:25.571170 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10028
I20260811 04:47:25.571306 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:25.571306 10106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1: Bootstrap starting.
I20260811 04:47:25.571681 23106 external_mini_cluster.cc:1428] Setting key 82a2e33abd778b6304500b88d8a60d5a
I20260811 04:47:25.574385 10106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:25.575254 10106 log.cc:824] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:25.577850 10106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1: No bootstrap required, opened a new log
Aug 11 04:47:25 dist-test-slave-ss4n krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:25.581799 10106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36615 } }
I20260811 04:47:25.582098 10106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:25.582280 10106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fae8f4ead874f7da0f1d03c84ce25a1, State: Initialized, Role: FOLLOWER
I20260811 04:47:25.582863 10106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36615 } }
I20260811 04:47:25.583040 10106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:25.583137 10106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:25.583278 10106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:25.584358 10106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36615 } }
I20260811 04:47:25.584818 10106 leader_election.cc:302] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fae8f4ead874f7da0f1d03c84ce25a1; no voters:
I20260811 04:47:25.585340 10106 leader_election.cc:288] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:25.585525 10112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:25.585819 10112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fae8f4ead874f7da0f1d03c84ce25a1, State: Running, Role: LEADER
I20260811 04:47:25.586287 10112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36615 } }
I20260811 04:47:25.587046 10106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:25.588402 10114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fae8f4ead874f7da0f1d03c84ce25a1. Latest consensus state: current_term: 1 leader_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36615 } } }
I20260811 04:47:25.588498 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:25.573363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55884 (local address 127.22.144.190:36615)
0811 04:47:25.573930 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:25.573956 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:25.573999 (+ 43us) server_negotiation.cc:407] Connection header received
0811 04:47:25.574856 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:25.574879 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:25.575291 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:25.575726 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:25.576705 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:25.577870 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:25.578808 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:25.579324 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:25.579512 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:25.582253 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:25.582259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:25.582275 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:25.582310 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:25.584461 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:25.585105 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:25.585112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:25.585119 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:25.585301 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:25.585796 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:25.585800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:25.585802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:25.586444 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:25.586681 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:25.586987 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 04:47:25.587324 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":163,"threads_started":1}
I20260811 04:47:25.588544 10114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:25.588886 10113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fae8f4ead874f7da0f1d03c84ce25a1" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 36615 } } }
I20260811 04:47:25.588970 10113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:25.590898 10121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:25.593989 10121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:25.601153 10121 catalog_manager.cc:1347] Generated new cluster ID: b8e5a42fdc084fda9d91070ae6638c6f
I20260811 04:47:25.601279 10121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:25.609555 10121 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:25.610988 10121 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:25.621541 10121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1: Generated new TSK 0
I20260811 04:47:25.622530 10121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:47:25.658679 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36615
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:25.783471 10136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:25.783802 10136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:25.783849 10136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:25.788161 10136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:25.788242 10136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:25.788314 10136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:25.794061 10136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:25.795208 10136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:25.796258 10136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:25.806984 10142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:25.806994 10146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:25.807049 10136 server_base.cc:1034] running on GCE node
W20260811 04:47:25.806975 10143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:25.807682 10136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:25.808401 10136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:25.809587 10136 hybrid_clock.cc:648] HybridClock initialized: now 1786423645809547 us; error 51 us; skew 500 ppm
Aug 11 04:47:25 dist-test-slave-ss4n krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423645, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:25.812639 10136 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:47:25.813742 10136 webserver.cc:460] Webserver started at http://127.22.144.129:40391/ using document root <none> and password file <none>
I20260811 04:47:25.814353 10136 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:25.814422 10136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:25.814697 10136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:25.817131 10136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "9f8ff372c0bd42888243fb19cb795bc2"
format_stamp: "Formatted at 2026-08-11 04:47:25 on dist-test-slave-ss4n"
server_key: "50f838ab7f0744c4394ff329e2945db8"
server_key_iv: "512e46cc753422ce896ba9636dfecab3"
server_key_version: "encryptionkey@0"
I20260811 04:47:25.817862 10136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "9f8ff372c0bd42888243fb19cb795bc2"
format_stamp: "Formatted at 2026-08-11 04:47:25 on dist-test-slave-ss4n"
server_key: "50f838ab7f0744c4394ff329e2945db8"
server_key_iv: "512e46cc753422ce896ba9636dfecab3"
server_key_version: "encryptionkey@0"
I20260811 04:47:25.822213 10136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:47:25.825229 10153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:25.826673 10136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:25.826830 10136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "9f8ff372c0bd42888243fb19cb795bc2"
format_stamp: "Formatted at 2026-08-11 04:47:25 on dist-test-slave-ss4n"
server_key: "50f838ab7f0744c4394ff329e2945db8"
server_key_iv: "512e46cc753422ce896ba9636dfecab3"
server_key_version: "encryptionkey@0"
I20260811 04:47:25.826982 10136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:25.850749 10136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:25.854583 10136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:25.854825 10136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:25.855500 10136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:25.856488 10136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:25.856564 10136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:25.856643 10136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:25.856690 10136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:25.866093 10136 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:43681
I20260811 04:47:25.866173 10280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:43681 every 8 connection(s)
I20260811 04:47:25.867187 10136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
Aug 11 04:47:25 dist-test-slave-ss4n krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423645, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:25.877888 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10136
I20260811 04:47:25.878074 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:47:25.878543 23106 external_mini_cluster.cc:1428] Setting key 7ad21281552d6eee1365d903c8be7792
I20260811 04:47:25.883322 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:25.869314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:44213 (local address 127.22.144.190:36615)
0811 04:47:25.869516 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:25.869523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:25.870470 (+ 947us) server_negotiation.cc:407] Connection header received
0811 04:47:25.871895 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:25.871899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:25.871960 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:25.872041 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:25.873282 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:25.873925 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:25.875140 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:25.875421 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:25.875538 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:25.878547 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:25.878551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:25.878553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:25.878582 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:25.880806 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:25.881410 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:25.881418 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:25.881423 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:25.881565 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:25.881955 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:25.881961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:25.881966 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:25.882272 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:25.882384 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:25.882930 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 04:47:25.883094 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 04:47:25.884218 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:25.869769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36615 (local address 127.22.144.129:44213)
0811 04:47:25.870291 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:25.870340 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:25.871652 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:25.872228 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:25.872235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:25.872604 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:25.873109 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:25.873123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:25.874070 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:25.874073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:25.875000 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:25.875010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:25.875602 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:25.875606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:25.875789 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:25.876447 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:25.876464 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:25.878329 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:25.880954 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:25.880960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:25.880976 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:25.881299 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:25.881684 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:25.881688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:25.881690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:25.881855 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:25.882386 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:25.882391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:25.882682 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 04:47:25.882858 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 04:47:25.883153 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":118,"threads_started":1}
I20260811 04:47:25.885710 10282 heartbeater.cc:347] Connected to a master server at 127.22.144.190:36615
I20260811 04:47:25.885988 10282 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:25.886613 10282 heartbeater.cc:511] Master 127.22.144.190:36615 requested a full tablet report, sending...
I20260811 04:47:25.888605 10066 ts_manager.cc:194] Registered new tserver with Master: 9f8ff372c0bd42888243fb19cb795bc2 (127.22.144.129:43681)
I20260811 04:47:25.890138 10066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:44213
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:47:25.913863 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36615
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:26.039776 10290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:26.040171 10290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:26.040256 10290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:26.044792 10290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:26.044910 10290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:26.045011 10290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:26.050909 10290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:26.052203 10290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:26.053433 10290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:26.064563 10303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:26.064563 10297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:26.064570 10298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:26.065003 10290 server_base.cc:1034] running on GCE node
I20260811 04:47:26.065457 10290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:26.066265 10290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:26.067488 10290 hybrid_clock.cc:648] HybridClock initialized: now 1786423646067453 us; error 47 us; skew 500 ppm
Aug 11 04:47:26 dist-test-slave-ss4n krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:26.070765 10290 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:47:26.071779 10290 webserver.cc:460] Webserver started at http://127.22.144.130:45635/ using document root <none> and password file <none>
I20260811 04:47:26.072412 10290 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:26.072515 10290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:26.072772 10290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:26.075054 10290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "fbbd04fd1be8416983124c2cb1607650"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "366ab8362ebe0e29164e29d2a695a7ac"
server_key_iv: "652fa5481b0779cc5e355711f44d0d20"
server_key_version: "encryptionkey@0"
I20260811 04:47:26.075691 10290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "fbbd04fd1be8416983124c2cb1607650"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "366ab8362ebe0e29164e29d2a695a7ac"
server_key_iv: "652fa5481b0779cc5e355711f44d0d20"
server_key_version: "encryptionkey@0"
I20260811 04:47:26.080055 10290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:26.082909 10311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:26.084435 10290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:47:26.084594 10290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "fbbd04fd1be8416983124c2cb1607650"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "366ab8362ebe0e29164e29d2a695a7ac"
server_key_iv: "652fa5481b0779cc5e355711f44d0d20"
server_key_version: "encryptionkey@0"
I20260811 04:47:26.084735 10290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:26.114354 10290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:26.117578 10290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:26.117818 10290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:26.118464 10290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:26.119401 10290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:26.119477 10290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:26.119558 10290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:26.119612 10290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:26.129204 10290 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:38205
I20260811 04:47:26.129225 10437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:38205 every 8 connection(s)
I20260811 04:47:26.130260 10290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:26.134801 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10290
I20260811 04:47:26.134950 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:47:26.135252 23106 external_mini_cluster.cc:1428] Setting key 1c40921c049424033c6403f88cbf8d86
Aug 11 04:47:26 dist-test-slave-ss4n krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:26.145311 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.132268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:34667 (local address 127.22.144.190:36615)
0811 04:47:26.132428 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.132432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.133366 (+ 934us) server_negotiation.cc:407] Connection header received
0811 04:47:26.134959 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.134962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.135028 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.135136 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:26.136601 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.137135 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.138443 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.138674 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.138787 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.141459 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:26.141462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:26.141464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:26.141535 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:26.143094 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:26.143676 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:26.143679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:26.143680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:26.143758 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:26.144120 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:26.144123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:26.144125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:26.144326 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:26.144438 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.144990 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.145135 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:47:26.146289 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.132702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36615 (local address 127.22.144.130:34667)
0811 04:47:26.133206 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.133252 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.134730 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.135365 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.135375 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.135907 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:26.136450 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.136465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.137275 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.137284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.138303 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.138314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.138850 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.138854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.139032 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.139709 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:26.139727 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:26.141268 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:26.143240 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:26.143247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:26.143264 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:26.143546 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:26.143856 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:26.143860 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:26.143862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:26.144015 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:26.144439 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:26.144447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:26.144726 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.144926 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.145299 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":121,"threads_started":1}
I20260811 04:47:26.147677 10438 heartbeater.cc:347] Connected to a master server at 127.22.144.190:36615
I20260811 04:47:26.147964 10438 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:26.148516 10438 heartbeater.cc:511] Master 127.22.144.190:36615 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:47:26.149782 10066 ts_manager.cc:194] Registered new tserver with Master: fbbd04fd1be8416983124c2cb1607650 (127.22.144.130:38205)
I20260811 04:47:26.150408 10066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:34667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:47:26.170042 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36615
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:26.292858 10445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:26.293172 10445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:26.293218 10445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:26.297379 10445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:26.297451 10445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:26.297540 10445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:26.303274 10445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:36615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:26.304529 10445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:26.305815 10445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:26.316424 10456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:26.316424 10451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:26.316425 10453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:26.316728 10445 server_base.cc:1034] running on GCE node
I20260811 04:47:26.317390 10445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:26.318181 10445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:26.319355 10445 hybrid_clock.cc:648] HybridClock initialized: now 1786423646319326 us; error 41 us; skew 500 ppm
Aug 11 04:47:26 dist-test-slave-ss4n krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:26.322495 10445 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:47:26.323509 10445 webserver.cc:460] Webserver started at http://127.22.144.131:45139/ using document root <none> and password file <none>
I20260811 04:47:26.324138 10445 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:26.324242 10445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:26.324494 10445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:26.326859 10445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "6091ec9c22a049809e761cfbb5a18a2f"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "00bd85a6383337663b5c62a58441c931"
server_key_iv: "7ce18970cce4f3b47a6bf4eb9587dc9e"
server_key_version: "encryptionkey@0"
I20260811 04:47:26.327486 10445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "6091ec9c22a049809e761cfbb5a18a2f"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "00bd85a6383337663b5c62a58441c931"
server_key_iv: "7ce18970cce4f3b47a6bf4eb9587dc9e"
server_key_version: "encryptionkey@0"
I20260811 04:47:26.331807 10445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:26.334403 10466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:26.335729 10445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:26.335871 10445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "6091ec9c22a049809e761cfbb5a18a2f"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "00bd85a6383337663b5c62a58441c931"
server_key_iv: "7ce18970cce4f3b47a6bf4eb9587dc9e"
server_key_version: "encryptionkey@0"
I20260811 04:47:26.336001 10445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:26.359602 10445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:26.363322 10445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:26.363560 10445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:26.364295 10445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:26.365226 10445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:26.365275 10445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:26.365351 10445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:26.365384 10445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:26.376175 10445 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:39913
I20260811 04:47:26.376199 10608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:39913 every 8 connection(s)
I20260811 04:47:26.377252 10445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:26.378778 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10445
I20260811 04:47:26.378931 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:47:26.379359 23106 external_mini_cluster.cc:1428] Setting key 2a97af8c12191d4c1176488fae6be31b
Aug 11 04:47:26 dist-test-slave-ss4n krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423646, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:26.393510 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.379459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:52319 (local address 127.22.144.190:36615)
0811 04:47:26.379598 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.379602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.380749 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 04:47:26.382361 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.382364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.382410 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.382480 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:26.383790 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.384321 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.385724 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.385959 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.386044 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.389285 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:26.389288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:26.389291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:26.389322 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:26.391063 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:26.391672 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:26.391674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:26.391676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:26.391782 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:26.392198 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:26.392202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:26.392204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:26.392428 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:26.392550 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.393169 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.393310 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 04:47:26.394589 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.380020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:36615 (local address 127.22.144.131:52319)
0811 04:47:26.380586 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.380649 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.382125 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.382669 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.382677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.383106 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:26.383640 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.383656 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.384445 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.384448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.385603 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.385614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.386122 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.386125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.386297 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.386901 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:26.386920 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:26.388949 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:26.391227 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:26.391233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:26.391251 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:26.391566 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:26.391886 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:26.391889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:26.391890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:26.392044 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:26.392551 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:26.392560 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:26.392885 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.393111 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.393549 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":167,"threads_started":1}
I20260811 04:47:26.395844 10609 heartbeater.cc:347] Connected to a master server at 127.22.144.190:36615
I20260811 04:47:26.396083 10609 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:26.396593 10609 heartbeater.cc:511] Master 127.22.144.190:36615 requested a full tablet report, sending...
I20260811 04:47:26.397758 10066 ts_manager.cc:194] Registered new tserver with Master: 6091ec9c22a049809e761cfbb5a18a2f (127.22.144.131:39913)
I20260811 04:47:26.398339 10066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:52319
I20260811 04:47:26.405989 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:47:26.417750 10109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.408292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.22.144.190:36615)
0811 04:47:26.408476 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.408481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.408634 (+ 153us) server_negotiation.cc:407] Connection header received
0811 04:47:26.408773 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.408776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.408824 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.408903 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:26.409753 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.410393 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.411271 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.411511 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.411635 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.413753 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:26.413760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:26.413769 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:26.413800 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:26.415290 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:26.416073 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:26.416077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:26.416079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:26.416180 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:26.416578 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:26.416582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:26.416585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:26.416852 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:26.417041 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.417387 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.417549 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 04:47:26.421358 10066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:47:26.424819 10066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:47:26.443215 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.436076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.22.144.130:38205)
0811 04:47:26.436691 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.436708 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.437023 (+ 315us) server_negotiation.cc:407] Connection header received
0811 04:47:26.437103 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.437108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.437320 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.437524 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:26.437828 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.439078 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.441371 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.442286 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.442376 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.442550 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.442918 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.443034 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":145,"threads_started":1}
I20260811 04:47:26.443228 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.436611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.22.144.129:43681)
0811 04:47:26.437856 (+ 1245us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.437863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.437889 (+ 26us) server_negotiation.cc:407] Connection header received
0811 04:47:26.437977 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.437986 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.438159 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.438332 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:26.438571 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.439508 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.441279 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.442351 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.442520 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.442692 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.442893 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.443010 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":100,"threads_started":1}
I20260811 04:47:26.443480 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.435511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:39913 (local address 127.0.0.1:52010)
0811 04:47:26.436648 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.436709 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.436876 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.437726 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.437729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.437751 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:26.437850 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.437857 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.439254 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.439258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.441214 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.441224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.442910 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.442913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.443024 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.443048 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.443128 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.443295 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":243,"thread_start_us":123,"threads_started":1}
I20260811 04:47:26.444033 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.435646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38205 (local address 127.0.0.1:56102)
0811 04:47:26.436673 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.436709 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.436864 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.437539 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.437546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.437578 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:26.437700 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.437706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.439273 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.439277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.441119 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.441135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.442654 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.442668 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.442751 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.442775 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.443797 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.443889 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":178,"thread_start_us":134,"threads_started":1}
I20260811 04:47:26.444084 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.436355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43681 (local address 127.0.0.1:49940)
0811 04:47:26.436839 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.436854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.436947 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.438310 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.438313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.438333 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:26.438425 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.438434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.439658 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.439662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.441112 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.441127 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.442519 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.442522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.442652 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.442698 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.442831 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.443953 (+ 1122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":30,"thread_start_us":121,"threads_started":1}
I20260811 04:47:26.444152 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.435628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.22.144.131:39913)
0811 04:47:26.436458 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.436471 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.436811 (+ 340us) server_negotiation.cc:407] Connection header received
0811 04:47:26.437081 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.437089 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.437321 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.437471 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:26.437984 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.439099 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.441450 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.442540 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.442643 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.442758 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.443879 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.443992 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":156,"threads_started":1}
I20260811 04:47:26.447177 10210 tablet_service.cc:1467] Processing CreateTablet for tablet efdf6921c0584c62a8c8a00110d6877d (DEFAULT_TABLE table=test-table [id=04d35d2dd0174902ac852c4f221f83eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:26.447377 10364 tablet_service.cc:1467] Processing CreateTablet for tablet efdf6921c0584c62a8c8a00110d6877d (DEFAULT_TABLE table=test-table [id=04d35d2dd0174902ac852c4f221f83eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:26.447377 10525 tablet_service.cc:1467] Processing CreateTablet for tablet efdf6921c0584c62a8c8a00110d6877d (DEFAULT_TABLE table=test-table [id=04d35d2dd0174902ac852c4f221f83eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:26.448416 10210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdf6921c0584c62a8c8a00110d6877d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:26.448503 10525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdf6921c0584c62a8c8a00110d6877d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:26.448495 10364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdf6921c0584c62a8c8a00110d6877d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:26.455394 10630 tablet_bootstrap.cc:492] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: Bootstrap starting.
I20260811 04:47:26.456540 10631 tablet_bootstrap.cc:492] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Bootstrap starting.
I20260811 04:47:26.457446 10632 tablet_bootstrap.cc:492] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Bootstrap starting.
I20260811 04:47:26.458921 10631 tablet_bootstrap.cc:654] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:26.458921 10630 tablet_bootstrap.cc:654] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:26.459750 10632 tablet_bootstrap.cc:654] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:26.460021 10630 log.cc:824] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:26.460222 10631 log.cc:824] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:26.461197 10632 log.cc:824] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:26.462628 10630 tablet_bootstrap.cc:492] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: No bootstrap required, opened a new log
I20260811 04:47:26.462841 10631 tablet_bootstrap.cc:492] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: No bootstrap required, opened a new log
I20260811 04:47:26.462881 10630 ts_tablet_manager.cc:1397] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 04:47:26.463063 10631 ts_tablet_manager.cc:1397] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 04:47:26.463400 10632 tablet_bootstrap.cc:492] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: No bootstrap required, opened a new log
I20260811 04:47:26.463593 10632 ts_tablet_manager.cc:1397] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 04:47:26.467918 10630 raft_consensus.cc:348] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.467943 10632 raft_consensus.cc:348] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.467942 10631 raft_consensus.cc:348] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.468246 10631 raft_consensus.cc:374] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:26.468246 10632 raft_consensus.cc:374] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:26.468248 10630 raft_consensus.cc:374] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:26.468315 10630 raft_consensus.cc:729] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f8ff372c0bd42888243fb19cb795bc2, State: Initialized, Role: FOLLOWER
I20260811 04:47:26.468317 10631 raft_consensus.cc:729] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbbd04fd1be8416983124c2cb1607650, State: Initialized, Role: FOLLOWER
I20260811 04:47:26.468327 10632 raft_consensus.cc:729] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6091ec9c22a049809e761cfbb5a18a2f, State: Initialized, Role: FOLLOWER
I20260811 04:47:26.468868 10630 consensus_queue.cc:260] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.468873 10631 consensus_queue.cc:260] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.468873 10632 consensus_queue.cc:260] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.469933 10438 heartbeater.cc:503] Master 127.22.144.190:36615 was elected leader, sending a full tablet report...
I20260811 04:47:26.470235 10282 heartbeater.cc:503] Master 127.22.144.190:36615 was elected leader, sending a full tablet report...
I20260811 04:47:26.470316 10609 heartbeater.cc:503] Master 127.22.144.190:36615 was elected leader, sending a full tablet report...
I20260811 04:47:26.471390 10630 ts_tablet_manager.cc:1428] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 04:47:26.471414 10632 ts_tablet_manager.cc:1428] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Time spent starting tablet: real 0.008s user 0.005s sys 0.003s
I20260811 04:47:26.470163 10631 ts_tablet_manager.cc:1428] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260811 04:47:26.618801 10284 tablet.cc:2369] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:47:26.628401 10610 tablet.cc:2369] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:47:26.631716 10439 tablet.cc:2369] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:47:26.667977 10638 raft_consensus.cc:482] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:26.668254 10638 raft_consensus.cc:504] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.669665 10638 leader_election.cc:288] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbbd04fd1be8416983124c2cb1607650 (127.22.144.130:38205), 9f8ff372c0bd42888243fb19cb795bc2 (127.22.144.129:43681)
I20260811 04:47:26.674227 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.669940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:38205 (local address 127.22.144.131:39467)
0811 04:47:26.670089 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.670107 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.670218 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.670506 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.670510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.670531 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:26.670657 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.670666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.671698 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.671701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.672829 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.672839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.673703 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.673706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.673827 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.673861 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.673964 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.674047 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260811 04:47:26.674372 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.670055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:39467 (local address 127.22.144.130:38205)
0811 04:47:26.670203 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.670206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.670247 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:47:26.670316 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.670319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.670378 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.670494 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:26.670817 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.671574 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.672963 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.673566 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.673694 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.673751 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.674158 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.674249 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 04:47:26.674811 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.670280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43681 (local address 127.22.144.131:39311)
0811 04:47:26.670703 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.670715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:26.670846 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:26.671101 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:26.671103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:26.671122 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:26.671250 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.671256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.672251 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.672254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.673295 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:26.673306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.674366 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.674372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:26.674497 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.674519 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 04:47:26.674592 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 04:47:26.674663 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":158,"threads_started":1}
I20260811 04:47:26.674824 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.670482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:39311 (local address 127.22.144.129:43681)
0811 04:47:26.670750 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.670755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.670805 (+ 50us) server_negotiation.cc:407] Connection header received
0811 04:47:26.670954 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.670956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.671000 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.671062 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:26.671367 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.672061 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.673431 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.674065 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.674222 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.674278 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.674618 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.674718 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 04:47:26.675050 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdf6921c0584c62a8c8a00110d6877d" candidate_uuid: "6091ec9c22a049809e761cfbb5a18a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbbd04fd1be8416983124c2cb1607650" is_pre_election: true
I20260811 04:47:26.675457 10385 raft_consensus.cc:2454] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6091ec9c22a049809e761cfbb5a18a2f in term 0.
I20260811 04:47:26.675629 10230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdf6921c0584c62a8c8a00110d6877d" candidate_uuid: "6091ec9c22a049809e761cfbb5a18a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8ff372c0bd42888243fb19cb795bc2" is_pre_election: true
I20260811 04:47:26.675992 10230 raft_consensus.cc:2454] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6091ec9c22a049809e761cfbb5a18a2f in term 0.
I20260811 04:47:26.676051 10468 leader_election.cc:302] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6091ec9c22a049809e761cfbb5a18a2f, fbbd04fd1be8416983124c2cb1607650; no voters:
I20260811 04:47:26.676422 10638 raft_consensus.cc:2781] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:47:26.676508 10638 raft_consensus.cc:482] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:47:26.676539 10638 raft_consensus.cc:3037] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:26.677723 10638 raft_consensus.cc:504] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.678123 10638 leader_election.cc:288] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [CANDIDATE]: Term 1 election: Requested vote from peers fbbd04fd1be8416983124c2cb1607650 (127.22.144.130:38205), 9f8ff372c0bd42888243fb19cb795bc2 (127.22.144.129:43681)
I20260811 04:47:26.678737 10230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdf6921c0584c62a8c8a00110d6877d" candidate_uuid: "6091ec9c22a049809e761cfbb5a18a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f8ff372c0bd42888243fb19cb795bc2"
I20260811 04:47:26.678737 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdf6921c0584c62a8c8a00110d6877d" candidate_uuid: "6091ec9c22a049809e761cfbb5a18a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbbd04fd1be8416983124c2cb1607650"
I20260811 04:47:26.678856 10230 raft_consensus.cc:3037] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:26.678900 10385 raft_consensus.cc:3037] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:26.680035 10230 raft_consensus.cc:2454] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6091ec9c22a049809e761cfbb5a18a2f in term 1.
I20260811 04:47:26.680073 10385 raft_consensus.cc:2454] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6091ec9c22a049809e761cfbb5a18a2f in term 1.
I20260811 04:47:26.680406 10468 leader_election.cc:302] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6091ec9c22a049809e761cfbb5a18a2f, 9f8ff372c0bd42888243fb19cb795bc2; no voters:
I20260811 04:47:26.680655 10638 raft_consensus.cc:2781] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:26.680931 10638 raft_consensus.cc:686] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 1 LEADER]: Becoming Leader. State: Replica: 6091ec9c22a049809e761cfbb5a18a2f, State: Running, Role: LEADER
I20260811 04:47:26.681409 10638 consensus_queue.cc:237] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } }
I20260811 04:47:26.685199 10064 catalog_manager.cc:5495] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6091ec9c22a049809e761cfbb5a18a2f (127.22.144.131). New cstate: current_term: 1 leader_uuid: "6091ec9c22a049809e761cfbb5a18a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6091ec9c22a049809e761cfbb5a18a2f" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 39913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 } health_report { overall_health: UNKNOWN } } }
I20260811 04:47:26.741415 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.737282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.22.144.131:39913)
0811 04:47:26.737442 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:26.737447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:26.737464 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:47:26.737580 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:26.737583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:26.737641 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:26.737770 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:47:26.738080 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.738598 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.739624 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:26.739848 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:26.739966 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:26.740422 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:47:26.740910 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:47:26.741090 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:26.741188 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 04:47:26.741231 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 04:47:26.748190 10385 raft_consensus.cc:1261] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 1 FOLLOWER]: Refusing update from remote peer 6091ec9c22a049809e761cfbb5a18a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:26.748183 10230 raft_consensus.cc:1261] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 1 FOLLOWER]: Refusing update from remote peer 6091ec9c22a049809e761cfbb5a18a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:26.749027 10641 consensus_queue.cc:1035] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f8ff372c0bd42888243fb19cb795bc2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:26.749271 10638 consensus_queue.cc:1035] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbbd04fd1be8416983124c2cb1607650" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:26.767448 10065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:26.767633 10065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:26.770501 10065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fae8f4ead874f7da0f1d03c84ce25a1: Sending DeleteTablet for 3 replicas of tablet efdf6921c0584c62a8c8a00110d6877d
I20260811 04:47:26.771487 10364 tablet_service.cc:1514] Processing DeleteTablet for tablet efdf6921c0584c62a8c8a00110d6877d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:26 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:56102
I20260811 04:47:26.771462 10525 tablet_service.cc:1514] Processing DeleteTablet for tablet efdf6921c0584c62a8c8a00110d6877d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:26 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:52010
I20260811 04:47:26.771534 10210 tablet_service.cc:1514] Processing DeleteTablet for tablet efdf6921c0584c62a8c8a00110d6877d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:26 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:49940
I20260811 04:47:26.772255 10657 tablet_replica.cc:330] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2: stopping tablet replica
I20260811 04:47:26.772486 10655 tablet_replica.cc:330] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: stopping tablet replica
I20260811 04:47:26.772517 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10136
I20260811 04:47:26.772745 10657 raft_consensus.cc:2229] T efdf6921c0584c62a8c8a00110d6877d P 9f8ff372c0bd42888243fb19cb795bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:26.772933 10655 raft_consensus.cc:2229] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:26.773375 10655 raft_consensus.cc:2258] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:26.774497 10656 tablet_replica.cc:330] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: stopping tablet replica
I20260811 04:47:26.775753 10655 ts_tablet_manager.cc:1905] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:26.776198 10656 raft_consensus.cc:2229] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:47:26.776743 10656 raft_consensus.cc:2258] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:26.779567 10655 ts_tablet_manager.cc:1918] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:26.779726 10655 log.cc:1196] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/efdf6921c0584c62a8c8a00110d6877d
I20260811 04:47:26.780061 10655 ts_tablet_manager.cc:1939] T efdf6921c0584c62a8c8a00110d6877d P fbbd04fd1be8416983124c2cb1607650: Deleting consensus metadata
I20260811 04:47:26.781949 10050 catalog_manager.cc:4841] TS fbbd04fd1be8416983124c2cb1607650 (127.22.144.130:38205): tablet efdf6921c0584c62a8c8a00110d6877d (table test-table [id=04d35d2dd0174902ac852c4f221f83eb]) successfully deleted
I20260811 04:47:26.783196 10656 ts_tablet_manager.cc:1905] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 04:47:26.785919 10050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:43681 (error 108)
I20260811 04:47:26.786129 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10290
I20260811 04:47:26.789820 10656 ts_tablet_manager.cc:1918] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:26.789991 10656 log.cc:1196] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/efdf6921c0584c62a8c8a00110d6877d
I20260811 04:47:26.790439 10656 ts_tablet_manager.cc:1939] T efdf6921c0584c62a8c8a00110d6877d P 6091ec9c22a049809e761cfbb5a18a2f: Deleting consensus metadata
I20260811 04:47:26.791155 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:26.790765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:43681 (local address 127.0.0.1:49946)
0811 04:47:26.790934 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:26.791032 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 04:47:26.791383 10050 catalog_manager.cc:4584] TS 9f8ff372c0bd42888243fb19cb795bc2 (127.22.144.129:43681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet efdf6921c0584c62a8c8a00110d6877d: Network error: Client connection negotiation failed: client connection to 127.22.144.129:43681: connect: Connection refused (error 111)
I20260811 04:47:26.792032 10052 catalog_manager.cc:4841] TS 6091ec9c22a049809e761cfbb5a18a2f (127.22.144.131:39913): tablet efdf6921c0584c62a8c8a00110d6877d (table test-table [id=04d35d2dd0174902ac852c4f221f83eb]) successfully deleted
I20260811 04:47:26.798358 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10445
I20260811 04:47:26.807215 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10028
2026-08-11T04:47:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2733 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T04:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:26Z Disabled control of system clock
I20260811 04:47:26.845894 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39161
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:39161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 04:47:26.968958 10664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:26.969278 10664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:26.969318 10664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:26.973572 10664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:26.973644 10664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:26.973660 10664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:26.973677 10664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:26.973693 10664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:26.978600 10664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:39161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:39161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:26.979856 10664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:26.981377 10664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:47:26.990970 10664 server_base.cc:1034] running on GCE node
W20260811 04:47:26.990871 10671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:26.990897 10670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:26.990897 10673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:26.991914 10664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:26.993119 10664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:26.994313 10664 hybrid_clock.cc:648] HybridClock initialized: now 1786423646994244 us; error 85 us; skew 500 ppm
I20260811 04:47:26.996310 10664 webserver.cc:460] Webserver started at http://127.22.144.190:40101/ using document root <none> and password file <none>
I20260811 04:47:26.996933 10664 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:26.997033 10664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:26.997277 10664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:26.999531 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "1976f6faf0e042559411ef06644a2814"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "72f3daf4b2703846156b772b7b2e1598"
server_key_iv: "36e234650764f5e6008489ff06e1db56"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.000223 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "1976f6faf0e042559411ef06644a2814"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "72f3daf4b2703846156b772b7b2e1598"
server_key_iv: "36e234650764f5e6008489ff06e1db56"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.004380 10664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:47:27.007197 10682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.008508 10664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:47:27.008690 10664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "1976f6faf0e042559411ef06644a2814"
format_stamp: "Formatted at 2026-08-11 04:47:26 on dist-test-slave-ss4n"
server_key: "72f3daf4b2703846156b772b7b2e1598"
server_key_iv: "36e234650764f5e6008489ff06e1db56"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.008848 10664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:27.025055 10664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:27.025889 10664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:27.026111 10664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:27.033712 10664 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:39161
I20260811 04:47:27.033725 10742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:39161 every 8 connection(s)
I20260811 04:47:27.034809 10664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:27.038612 10743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:27.043881 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10664
I20260811 04:47:27.044056 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:27.044436 23106 external_mini_cluster.cc:1428] Setting key 58d9f0de985a126c3f415d0151043fb2
I20260811 04:47:27.045596 10743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814: Bootstrap starting.
I20260811 04:47:27.048166 10743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:27.049121 10743 log.cc:824] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:27.051869 10743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814: No bootstrap required, opened a new log
I20260811 04:47:27.054922 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.046065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.22.144.190:39161)
0811 04:47:27.046773 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.046791 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.046831 (+ 40us) server_negotiation.cc:407] Connection header received
0811 04:47:27.047490 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.047508 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.047939 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.048235 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:27.048708 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.049963 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.051344 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.051853 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.052023 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.052949 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:27.052955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:27.052969 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:27.053392 (+ 423us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:27.053455 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:27.053476 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:27.053621 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.053799 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.054140 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":150,"threads_started":1}
I20260811 04:47:27.056596 10743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1976f6faf0e042559411ef06644a2814" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39161 } }
I20260811 04:47:27.056854 10743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:27.056923 10743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1976f6faf0e042559411ef06644a2814, State: Initialized, Role: FOLLOWER
I20260811 04:47:27.057525 10743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1976f6faf0e042559411ef06644a2814" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39161 } }
I20260811 04:47:27.057685 10743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:27.057776 10743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:27.057909 10743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:27.059029 10743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1976f6faf0e042559411ef06644a2814" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39161 } }
I20260811 04:47:27.059424 10743 leader_election.cc:302] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1976f6faf0e042559411ef06644a2814; no voters:
I20260811 04:47:27.059836 10743 leader_election.cc:288] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:27.059952 10749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:27.060174 10749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [term 1 LEADER]: Becoming Leader. State: Replica: 1976f6faf0e042559411ef06644a2814, State: Running, Role: LEADER
I20260811 04:47:27.060525 10749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1976f6faf0e042559411ef06644a2814" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39161 } }
I20260811 04:47:27.061372 10743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:27.061866 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1976f6faf0e042559411ef06644a2814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1976f6faf0e042559411ef06644a2814" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39161 } } }
I20260811 04:47:27.062022 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:27.062498 10754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:27.062441 10750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1976f6faf0e042559411ef06644a2814. Latest consensus state: current_term: 1 leader_uuid: "1976f6faf0e042559411ef06644a2814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1976f6faf0e042559411ef06644a2814" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 39161 } } }
I20260811 04:47:27.062587 10750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:27.065871 10754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:27.073114 10754 catalog_manager.cc:1347] Generated new cluster ID: 4694cb2e094942d8b1fdd7129704f6d9
I20260811 04:47:27.073226 10754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:27.088478 10754 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:27.089571 10754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:27.096381 10754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814: Generated new TSK 0
I20260811 04:47:27.097203 10754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:27.110569 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39161
--builtin_ntp_servers=127.22.144.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 04:47:27.232278 10772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:27.232587 10772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:27.232630 10772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:27.237046 10772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:27.237130 10772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:27.237193 10772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:27.242528 10772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:27.243781 10772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:27.245313 10772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:27.255913 10778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:27.256078 10772 server_base.cc:1034] running on GCE node
W20260811 04:47:27.255913 10777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:27.255913 10780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:27.256693 10772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:27.257637 10772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:27.258876 10772 hybrid_clock.cc:648] HybridClock initialized: now 1786423647258868 us; error 43 us; skew 500 ppm
I20260811 04:47:27.261546 10772 webserver.cc:460] Webserver started at http://127.22.144.129:34915/ using document root <none> and password file <none>
I20260811 04:47:27.262282 10772 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:27.262396 10772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:27.262642 10772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:27.265100 10772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "950474f19fdc4da6b27084a1a297c3d2"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "ddd5684da51e21aaedd83488527ce85b"
server_key_iv: "7d008dca05ab6d6c882810fe1eef84ae"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.265749 10772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "950474f19fdc4da6b27084a1a297c3d2"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "ddd5684da51e21aaedd83488527ce85b"
server_key_iv: "7d008dca05ab6d6c882810fe1eef84ae"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.270026 10772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 04:47:27.272673 10786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.274072 10772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:27.274224 10772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "950474f19fdc4da6b27084a1a297c3d2"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "ddd5684da51e21aaedd83488527ce85b"
server_key_iv: "7d008dca05ab6d6c882810fe1eef84ae"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.274358 10772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:27.306124 10772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:27.306963 10772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:27.307204 10772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:27.307858 10772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:27.308904 10772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:27.308979 10772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.309056 10772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:27.309098 10772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.318782 10772 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:39713
I20260811 04:47:27.318872 10921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:39713 every 8 connection(s)
I20260811 04:47:27.319900 10772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:47:27.329738 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10772
I20260811 04:47:27.329926 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:47:27.330257 23106 external_mini_cluster.cc:1428] Setting key f7ff42678f340b80c7f21ea27856c271
I20260811 04:47:27.331077 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.322353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:40489 (local address 127.22.144.190:39161)
0811 04:47:27.322510 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.322514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.323810 (+ 1296us) server_negotiation.cc:407] Connection header received
0811 04:47:27.325373 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.325377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.325437 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.325591 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:27.326657 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.327257 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.328471 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.328732 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.328832 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.330460 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:27.330464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:27.330467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:27.330484 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:27.330512 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:27.330523 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:27.330603 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.330839 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.330905 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:47:27.332351 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.322852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39161 (local address 127.22.144.129:40489)
0811 04:47:27.323635 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.323703 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.325086 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.325780 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.325787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.325888 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:27.326475 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.326489 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.327387 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.327390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.328334 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.328344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.328906 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.328909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.329063 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.330090 (+ 1027us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:27.330092 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:27.330146 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:27.330231 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:27.330634 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:27.330644 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:27.330650 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:47:27.330829 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 04:47:27.331200 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":143,"threads_started":1}
I20260811 04:47:27.332676 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39161
--builtin_ntp_servers=127.22.144.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 04:47:27.333950 10922 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39161
I20260811 04:47:27.334661 10922 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:27.335397 10922 heartbeater.cc:511] Master 127.22.144.190:39161 requested a full tablet report, sending...
I20260811 04:47:27.337417 10701 ts_manager.cc:194] Registered new tserver with Master: 950474f19fdc4da6b27084a1a297c3d2 (127.22.144.129:39713)
I20260811 04:47:27.339357 10701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.129:40489
W20260811 04:47:27.453619 10925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:27.453909 10925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:27.453949 10925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:27.458051 10925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:27.458122 10925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:27.458186 10925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:27.463382 10925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:27.464633 10925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:27.466099 10925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:27.476500 10932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:27.476495 10931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:27.476492 10935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:27.476850 10925 server_base.cc:1034] running on GCE node
I20260811 04:47:27.477325 10925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:27.478147 10925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:27.479310 10925 hybrid_clock.cc:648] HybridClock initialized: now 1786423647479291 us; error 29 us; skew 500 ppm
I20260811 04:47:27.481627 10925 webserver.cc:460] Webserver started at http://127.22.144.130:44959/ using document root <none> and password file <none>
I20260811 04:47:27.482288 10925 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:27.482360 10925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:27.482649 10925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:27.484913 10925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "138746399f004b34aca0d0cbdca7fbb0"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "5b6cf32dc95c69089c2afa519722ed02"
server_key_iv: "35866bd75c5ad45a9973617412af8249"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.485620 10925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "138746399f004b34aca0d0cbdca7fbb0"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "5b6cf32dc95c69089c2afa519722ed02"
server_key_iv: "35866bd75c5ad45a9973617412af8249"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.490098 10925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:27.493067 10941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.494737 10925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:27.494915 10925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "138746399f004b34aca0d0cbdca7fbb0"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "5b6cf32dc95c69089c2afa519722ed02"
server_key_iv: "35866bd75c5ad45a9973617412af8249"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.495054 10925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:27.518404 10925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:27.519166 10925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:27.519390 10925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:27.520038 10925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:27.521051 10925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:27.521126 10925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.521195 10925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:27.521239 10925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.531088 10925 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:40431
I20260811 04:47:27.531127 11076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:40431 every 8 connection(s)
I20260811 04:47:27.532076 10925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:27.541857 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.534129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:41373 (local address 127.22.144.190:39161)
0811 04:47:27.534351 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.534355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.535536 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 04:47:27.536931 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.536934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.536986 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.537064 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:27.538161 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.538705 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.539889 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.540115 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.540215 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.541267 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:27.541271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:27.541273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:27.541286 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:27.541306 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:27.541314 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:27.541375 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.541593 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.541698 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 04:47:27.541873 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 10925
I20260811 04:47:27.542025 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:47:27.542385 23106 external_mini_cluster.cc:1428] Setting key 7146d907e3764322b600d07bbd08c728
I20260811 04:47:27.542850 11081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.534589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39161 (local address 127.22.144.130:41373)
0811 04:47:27.535382 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.535425 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.536731 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.537259 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.537266 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.537391 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:27.538008 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.538022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.538837 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.538840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.539755 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.539765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.540267 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.540269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.540413 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.541001 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:27.541002 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:27.541044 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:27.541111 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:27.541392 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:27.541399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:27.541404 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 04:47:27.541572 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 04:47:27.541864 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
I20260811 04:47:27.544162 11077 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39161
I20260811 04:47:27.544448 11077 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:27.544821 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39161
--builtin_ntp_servers=127.22.144.148:37409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 04:47:27.545085 11077 heartbeater.cc:511] Master 127.22.144.190:39161 requested a full tablet report, sending...
I20260811 04:47:27.546352 10701 ts_manager.cc:194] Registered new tserver with Master: 138746399f004b34aca0d0cbdca7fbb0 (127.22.144.130:40431)
I20260811 04:47:27.547113 10701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.130:41373
W20260811 04:47:27.667658 11082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:27.668010 11082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:27.668061 11082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:27.672412 11082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:27.672519 11082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:27.672618 11082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:27.677959 11082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:37409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:39161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:27.679330 11082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:27.680886 11082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:27.691756 11091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:27.691985 11082 server_base.cc:1034] running on GCE node
W20260811 04:47:27.691761 11088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:27.691756 11089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:27.692514 11082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:27.693287 11082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:27.694517 11082 hybrid_clock.cc:648] HybridClock initialized: now 1786423647694487 us; error 74 us; skew 500 ppm
I20260811 04:47:27.696769 11082 webserver.cc:460] Webserver started at http://127.22.144.131:39287/ using document root <none> and password file <none>
I20260811 04:47:27.697572 11082 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:27.697678 11082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:27.697942 11082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:27.700366 11082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "2f48bf128a65420bb21a773a3513a651"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "19a7381d05f9618a8cbd7d0cdb953183"
server_key_iv: "39c9cfa5e76313e1a21b1ca450e2f0a0"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.700982 11082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "2f48bf128a65420bb21a773a3513a651"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "19a7381d05f9618a8cbd7d0cdb953183"
server_key_iv: "39c9cfa5e76313e1a21b1ca450e2f0a0"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.705263 11082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 04:47:27.708323 11097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.709988 11082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:27.710145 11082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "2f48bf128a65420bb21a773a3513a651"
format_stamp: "Formatted at 2026-08-11 04:47:27 on dist-test-slave-ss4n"
server_key: "19a7381d05f9618a8cbd7d0cdb953183"
server_key_iv: "39c9cfa5e76313e1a21b1ca450e2f0a0"
server_key_version: "encryptionkey@0"
I20260811 04:47:27.710304 11082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:27.728582 11082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:27.729616 11082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:27.729875 11082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:27.730579 11082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:27.731639 11082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:27.731716 11082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.731786 11082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:27.731817 11082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:27.742004 11082 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:32995
I20260811 04:47:27.742039 11230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:32995 every 8 connection(s)
I20260811 04:47:27.743063 11082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:27.744434 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 11082
I20260811 04:47:27.744560 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:47:27.744889 23106 external_mini_cluster.cc:1428] Setting key 338d12372fd34ba0a6975726f1bf1ba9
I20260811 04:47:27.753422 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.745228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:35051 (local address 127.22.144.190:39161)
0811 04:47:27.745378 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.745382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.746820 (+ 1438us) server_negotiation.cc:407] Connection header received
0811 04:47:27.748378 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.748381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.748440 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.748524 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:27.749517 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.750067 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.751323 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.751570 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.751696 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.752873 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:27.752877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:27.752880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:27.752902 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:27.752924 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:27.752931 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:27.753010 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.753191 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.753266 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:47:27.754410 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.745831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:39161 (local address 127.22.144.131:35051)
0811 04:47:27.746672 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.746718 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.748159 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.748719 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.748726 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.748821 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:27.749347 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.749361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.750233 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.750236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.751181 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.751192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.751751 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.751755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.751936 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.752561 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:27.752562 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:27.752609 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:27.752701 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:27.753026 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:27.753044 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:27.753049 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 04:47:27.753185 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 04:47:27.753530 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":162,"threads_started":1}
I20260811 04:47:27.755681 11231 heartbeater.cc:347] Connected to a master server at 127.22.144.190:39161
I20260811 04:47:27.755942 11231 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:27.756511 11231 heartbeater.cc:511] Master 127.22.144.190:39161 requested a full tablet report, sending...
I20260811 04:47:27.757731 10701 ts_manager.cc:194] Registered new tserver with Master: 2f48bf128a65420bb21a773a3513a651 (127.22.144.131:32995)
I20260811 04:47:27.758700 10701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.131:35051
I20260811 04:47:27.759480 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:47:27.766227 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.762033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.22.144.190:39161)
0811 04:47:27.762255 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.762258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.762318 (+ 60us) server_negotiation.cc:407] Connection header received
0811 04:47:27.762496 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.762499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.762552 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.762622 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:27.762986 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.763556 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.764419 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.764628 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.764706 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.765724 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:27.765727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:27.765729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:27.765742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:27.765759 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:27.765767 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:27.765869 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.765979 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.766016 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 04:47:27.769829 10700 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:47:27.773043 10700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:47:27.789618 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.783002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39713 (local address 127.0.0.1:34642)
0811 04:47:27.783876 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.783939 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.784098 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.784899 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.784902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.784915 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:27.785116 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.785124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.786410 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.786413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.787661 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.787672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.788974 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.788977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.789091 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.789152 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 04:47:27.789267 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 04:47:27.789401 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":45,"thread_start_us":121,"threads_started":1}
I20260811 04:47:27.789645 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.783346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.22.144.129:39713)
0811 04:47:27.783718 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.783724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.784039 (+ 315us) server_negotiation.cc:407] Connection header received
0811 04:47:27.784277 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.784282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.784534 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.784693 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:27.785256 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.786279 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.787836 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.788569 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.788677 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.788862 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.789325 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.789440 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":81,"threads_started":1}
I20260811 04:47:27.789702 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.782946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.22.144.131:32995)
0811 04:47:27.783604 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.783621 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.784448 (+ 827us) server_negotiation.cc:407] Connection header received
0811 04:47:27.784669 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.784674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.784849 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.784972 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:27.785383 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.786622 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.788305 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.789057 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.789191 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.789361 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.789410 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.789535 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":160,"threads_started":1}
I20260811 04:47:27.789639 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.782794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:32995 (local address 127.0.0.1:48676)
0811 04:47:27.783767 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.784368 (+ 601us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.784517 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.785116 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.785119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.785134 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:27.785250 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.785257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.786783 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.786786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.788163 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.788176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.789182 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.789185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.789282 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.789309 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 04:47:27.789372 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 04:47:27.789440 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":119,"thread_start_us":107,"threads_started":1}
I20260811 04:47:27.791385 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.783459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:40431 (local address 127.0.0.1:43224)
0811 04:47:27.783905 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.783939 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.784084 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.784829 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.784838 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.784867 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:27.785116 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.785124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.786792 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.786795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.788184 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.788192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.790984 (+ 2792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.790988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.791075 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.791098 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 04:47:27.791188 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 04:47:27.791260 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":131,"thread_start_us":98,"threads_started":1}
I20260811 04:47:27.791905 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.783851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.22.144.130:40431)
0811 04:47:27.784319 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.784332 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.784359 (+ 27us) server_negotiation.cc:407] Connection header received
0811 04:47:27.784447 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.784452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.784653 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.784804 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:27.785272 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.786672 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.789950 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.790621 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.790756 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:27.790917 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:27.791645 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 04:47:27.791753 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":151,"threads_started":1}
I20260811 04:47:27.793090 10844 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5f8150b65044348043b66b7e7b99b9 (DEFAULT_TABLE table=test-table [id=3faec01d0d724e33b69e43488f9cae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:27.793785 11155 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5f8150b65044348043b66b7e7b99b9 (DEFAULT_TABLE table=test-table [id=3faec01d0d724e33b69e43488f9cae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:27.794209 10844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5f8150b65044348043b66b7e7b99b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:27.794790 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5f8150b65044348043b66b7e7b99b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:27.794638 10999 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5f8150b65044348043b66b7e7b99b9 (DEFAULT_TABLE table=test-table [id=3faec01d0d724e33b69e43488f9cae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:27.795608 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5f8150b65044348043b66b7e7b99b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:27.800781 11252 tablet_bootstrap.cc:492] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: Bootstrap starting.
I20260811 04:47:27.802093 11253 tablet_bootstrap.cc:492] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Bootstrap starting.
I20260811 04:47:27.803361 11252 tablet_bootstrap.cc:654] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:27.803367 11254 tablet_bootstrap.cc:492] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Bootstrap starting.
I20260811 04:47:27.804234 11253 tablet_bootstrap.cc:654] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:27.804337 11252 log.cc:824] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:27.805061 11253 log.cc:824] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:27.805820 11254 tablet_bootstrap.cc:654] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:27.806512 11252 tablet_bootstrap.cc:492] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: No bootstrap required, opened a new log
I20260811 04:47:27.806728 11252 ts_tablet_manager.cc:1397] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:47:27.806746 11254 log.cc:824] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:27.808424 11253 tablet_bootstrap.cc:492] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: No bootstrap required, opened a new log
I20260811 04:47:27.808545 11254 tablet_bootstrap.cc:492] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: No bootstrap required, opened a new log
I20260811 04:47:27.808614 11253 ts_tablet_manager.cc:1397] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 04:47:27.808761 11254 ts_tablet_manager.cc:1397] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 04:47:27.811270 11252 raft_consensus.cc:348] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.811563 11252 raft_consensus.cc:374] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:27.811625 11252 raft_consensus.cc:729] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 950474f19fdc4da6b27084a1a297c3d2, State: Initialized, Role: FOLLOWER
I20260811 04:47:27.812170 11252 consensus_queue.cc:260] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.812342 11254 raft_consensus.cc:348] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.812690 11254 raft_consensus.cc:374] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:27.812750 11254 raft_consensus.cc:729] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138746399f004b34aca0d0cbdca7fbb0, State: Initialized, Role: FOLLOWER
I20260811 04:47:27.813050 11253 raft_consensus.cc:348] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.813270 11252 ts_tablet_manager.cc:1428] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:47:27.813297 11253 raft_consensus.cc:374] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:27.813227 11254 consensus_queue.cc:260] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.813344 11253 raft_consensus.cc:729] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f48bf128a65420bb21a773a3513a651, State: Initialized, Role: FOLLOWER
I20260811 04:47:27.813683 10922 heartbeater.cc:503] Master 127.22.144.190:39161 was elected leader, sending a full tablet report...
I20260811 04:47:27.813761 11253 consensus_queue.cc:260] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.814363 11231 heartbeater.cc:503] Master 127.22.144.190:39161 was elected leader, sending a full tablet report...
I20260811 04:47:27.814325 11254 ts_tablet_manager.cc:1428] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 04:47:27.814637 11253 ts_tablet_manager.cc:1428] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 04:47:27.814718 11077 heartbeater.cc:503] Master 127.22.144.190:39161 was elected leader, sending a full tablet report...
W20260811 04:47:27.821267 10923 tablet.cc:2369] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:47:27.994211 11233 tablet.cc:2369] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:47:27.994772 11260 raft_consensus.cc:482] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:27.995012 11260 raft_consensus.cc:504] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:27.996347 11260 leader_election.cc:288] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f48bf128a65420bb21a773a3513a651 (127.22.144.131:32995), 950474f19fdc4da6b27084a1a297c3d2 (127.22.144.129:39713)
I20260811 04:47:28.000837 11081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.996658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39713 (local address 127.22.144.130:45235)
0811 04:47:27.996857 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.996876 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.997009 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.997517 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.997528 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.997557 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:27.997698 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.997706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.998540 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.998544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.999565 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.999576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:28.000335 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:28.000340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:28.000459 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:28.000480 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 04:47:28.000575 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 04:47:28.000659 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":27,"spinlock_wait_cycles":3456}
I20260811 04:47:28.000908 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.996741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:45235 (local address 127.22.144.129:39713)
0811 04:47:27.996902 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.996907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.996928 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:47:27.997200 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.997203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.997259 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.997385 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:27.997836 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.998397 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.999701 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:28.000191 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:28.000339 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:28.000401 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:28.000607 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 04:47:28.000685 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 04:47:28.000954 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.996915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:50335 (local address 127.22.144.131:32995)
0811 04:47:27.997032 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:27.997035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:27.997209 (+ 174us) server_negotiation.cc:407] Connection header received
0811 04:47:27.997374 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:27.997377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:27.997420 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:27.997538 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:27.997854 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.998483 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.999758 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:28.000221 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:28.000374 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:28.000428 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:28.000723 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 04:47:28.000795 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 04:47:28.001684 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:27.996717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:32995 (local address 127.22.144.130:50335)
0811 04:47:27.997098 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:27.997113 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:27.997240 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:27.997547 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:27.997551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:27.997570 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:27.997722 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.997728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:27.998600 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:27.998605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:27.999622 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:27.999633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:28.000377 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:28.000381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:28.000580 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:28.000601 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 04:47:28.000704 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 04:47:28.000779 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260811 04:47:28.001878 10868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5f8150b65044348043b66b7e7b99b9" candidate_uuid: "138746399f004b34aca0d0cbdca7fbb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950474f19fdc4da6b27084a1a297c3d2" is_pre_election: true
I20260811 04:47:28.002198 10868 raft_consensus.cc:2454] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 138746399f004b34aca0d0cbdca7fbb0 in term 0.
I20260811 04:47:28.002633 11181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5f8150b65044348043b66b7e7b99b9" candidate_uuid: "138746399f004b34aca0d0cbdca7fbb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f48bf128a65420bb21a773a3513a651" is_pre_election: true
I20260811 04:47:28.002753 10942 leader_election.cc:302] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138746399f004b34aca0d0cbdca7fbb0, 950474f19fdc4da6b27084a1a297c3d2; no voters:
I20260811 04:47:28.002964 11181 raft_consensus.cc:2454] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 138746399f004b34aca0d0cbdca7fbb0 in term 0.
I20260811 04:47:28.003165 11260 raft_consensus.cc:2781] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:47:28.003276 11260 raft_consensus.cc:482] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:47:28.003345 11260 raft_consensus.cc:3037] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:28.004487 11260 raft_consensus.cc:504] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:28.004899 11260 leader_election.cc:288] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [CANDIDATE]: Term 1 election: Requested vote from peers 2f48bf128a65420bb21a773a3513a651 (127.22.144.131:32995), 950474f19fdc4da6b27084a1a297c3d2 (127.22.144.129:39713)
I20260811 04:47:28.005365 11181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5f8150b65044348043b66b7e7b99b9" candidate_uuid: "138746399f004b34aca0d0cbdca7fbb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f48bf128a65420bb21a773a3513a651"
I20260811 04:47:28.005542 11181 raft_consensus.cc:3037] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:28.005388 10868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5f8150b65044348043b66b7e7b99b9" candidate_uuid: "138746399f004b34aca0d0cbdca7fbb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "950474f19fdc4da6b27084a1a297c3d2"
I20260811 04:47:28.005721 10868 raft_consensus.cc:3037] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:28.006737 11181 raft_consensus.cc:2454] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 138746399f004b34aca0d0cbdca7fbb0 in term 1.
I20260811 04:47:28.007004 10868 raft_consensus.cc:2454] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 138746399f004b34aca0d0cbdca7fbb0 in term 1.
I20260811 04:47:28.007156 10943 leader_election.cc:302] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138746399f004b34aca0d0cbdca7fbb0, 2f48bf128a65420bb21a773a3513a651; no voters:
I20260811 04:47:28.007382 11260 raft_consensus.cc:2781] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:28.007682 11260 raft_consensus.cc:686] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 1 LEADER]: Becoming Leader. State: Replica: 138746399f004b34aca0d0cbdca7fbb0, State: Running, Role: LEADER
I20260811 04:47:28.008068 11260 consensus_queue.cc:237] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } }
I20260811 04:47:28.011958 10700 catalog_manager.cc:5495] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 138746399f004b34aca0d0cbdca7fbb0 (127.22.144.130). New cstate: current_term: 1 leader_uuid: "138746399f004b34aca0d0cbdca7fbb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "138746399f004b34aca0d0cbdca7fbb0" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 40431 } health_report { overall_health: HEALTHY } } }
W20260811 04:47:28.033782 11080 tablet.cc:2369] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:47:28.037256 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:28.032505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.22.144.130:40431)
0811 04:47:28.032664 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:28.032668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:28.032685 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:47:28.032769 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:28.032771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:28.032834 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:28.032966 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:47:28.033249 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:28.034138 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:28.035536 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:28.035779 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:28.035887 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:28.036278 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:47:28.036688 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:47:28.036822 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:28.037000 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 04:47:28.037085 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 04:47:28.044020 10868 raft_consensus.cc:1261] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 1 FOLLOWER]: Refusing update from remote peer 138746399f004b34aca0d0cbdca7fbb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:28.044020 11181 raft_consensus.cc:1261] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 1 FOLLOWER]: Refusing update from remote peer 138746399f004b34aca0d0cbdca7fbb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:28.044909 11263 consensus_queue.cc:1035] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "950474f19fdc4da6b27084a1a297c3d2" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 39713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:28.045133 11260 consensus_queue.cc:1035] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f48bf128a65420bb21a773a3513a651" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:28.052940 11266 mvcc.cc:205] Tried to move back new op lower bound from 7317191262379663360 to 7317191262244397056. Current Snapshot: MvccSnapshot[applied={T|T < 7317191262379663360}]
I20260811 04:47:28.055305 11267 mvcc.cc:205] Tried to move back new op lower bound from 7317191262379663360 to 7317191262244397056. Current Snapshot: MvccSnapshot[applied={T|T < 7317191262379663360}]
I20260811 04:47:28.062971 10699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:28.063179 10699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:28.065850 10699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1976f6faf0e042559411ef06644a2814: Sending DeleteTablet for 3 replicas of tablet 1c5f8150b65044348043b66b7e7b99b9
I20260811 04:47:28.066749 10844 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c5f8150b65044348043b66b7e7b99b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:28 UTC) from {username='slave'} at 127.0.0.1:34642
I20260811 04:47:28.066784 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c5f8150b65044348043b66b7e7b99b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:28 UTC) from {username='slave'} at 127.0.0.1:48676
I20260811 04:47:28.067301 11277 tablet_replica.cc:330] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2: stopping tablet replica
I20260811 04:47:28.067633 10999 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c5f8150b65044348043b66b7e7b99b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:28 UTC) from {username='slave'} at 127.0.0.1:43224
I20260811 04:47:28.067859 11277 raft_consensus.cc:2229] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:28.068126 11279 tablet_replica.cc:330] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: stopping tablet replica
I20260811 04:47:28.068372 11277 raft_consensus.cc:2258] T 1c5f8150b65044348043b66b7e7b99b9 P 950474f19fdc4da6b27084a1a297c3d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:28.068415 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10772
I20260811 04:47:28.068432 11279 raft_consensus.cc:2229] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:47:28.070523 11279 raft_consensus.cc:2258] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:28.071942 11279 ts_tablet_manager.cc:1905] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:28.074661 11279 ts_tablet_manager.cc:1918] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:28.074782 11279 log.cc:1196] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/1c5f8150b65044348043b66b7e7b99b9
I20260811 04:47:28.075095 11279 ts_tablet_manager.cc:1939] T 1c5f8150b65044348043b66b7e7b99b9 P 138746399f004b34aca0d0cbdca7fbb0: Deleting consensus metadata
I20260811 04:47:28.076305 10685 catalog_manager.cc:4841] TS 138746399f004b34aca0d0cbdca7fbb0 (127.22.144.130:40431): tablet 1c5f8150b65044348043b66b7e7b99b9 (table test-table [id=3faec01d0d724e33b69e43488f9cae36]) successfully deleted
I20260811 04:47:28.077785 11278 tablet_replica.cc:330] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: stopping tablet replica
I20260811 04:47:28.078117 11278 raft_consensus.cc:2229] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:28.078461 11278 raft_consensus.cc:2258] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:28.079893 11278 ts_tablet_manager.cc:1905] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 04:47:28.082163 10683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:39713 (error 108)
I20260811 04:47:28.082348 11278 ts_tablet_manager.cc:1918] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:28.082476 11278 log.cc:1196] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/1c5f8150b65044348043b66b7e7b99b9
I20260811 04:47:28.082765 11278 ts_tablet_manager.cc:1939] T 1c5f8150b65044348043b66b7e7b99b9 P 2f48bf128a65420bb21a773a3513a651: Deleting consensus metadata
I20260811 04:47:28.083168 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10925
I20260811 04:47:28.084604 10684 catalog_manager.cc:4841] TS 2f48bf128a65420bb21a773a3513a651 (127.22.144.131:32995): tablet 1c5f8150b65044348043b66b7e7b99b9 (table test-table [id=3faec01d0d724e33b69e43488f9cae36]) successfully deleted
I20260811 04:47:28.085417 11250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:28.085019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:39713 (local address 127.0.0.1:34646)
0811 04:47:28.085205 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:28.085301 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:39713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 04:47:28.086006 10683 catalog_manager.cc:4584] TS 950474f19fdc4da6b27084a1a297c3d2 (127.22.144.129:39713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c5f8150b65044348043b66b7e7b99b9: Network error: Client connection negotiation failed: client connection to 127.22.144.129:39713: connect: Connection refused (error 111)
I20260811 04:47:28.091259 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 11082
I20260811 04:47:28.099906 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 10664
2026-08-11T04:47:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1290 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:28 dist-test-slave-ss4n krb5kdc[11284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:28 dist-test-slave-ss4n krb5kdc[11284](info): set up 2 sockets
Aug 11 04:47:28 dist-test-slave-ss4n krb5kdc[11284](info): commencing operation
krb5kdc: starting...
W20260811 04:47:29.242259 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:29 dist-test-slave-ss4n krb5kdc[11284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:29.385368 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:33347
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:38609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:33347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:29.512461 11298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:29.512760 11298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:29.512844 11298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:29.517086 11298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:29.517154 11298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:29.517174 11298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:29.517192 11298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:29.517242 11298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:29.523078 11298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:38609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:33347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:33347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:29.524253 11298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:29.525362 11298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:47:29.535249 11298 server_base.cc:1034] running on GCE node
W20260811 04:47:29.535215 11304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:29.535202 11306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:29.535202 11303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:29.536144 11298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:29.537393 11298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:29.538563 11298 hybrid_clock.cc:648] HybridClock initialized: now 1786423649538544 us; error 35 us; skew 500 ppm
Aug 11 04:47:29 dist-test-slave-ss4n krb5kdc[11284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:29.545033 11298 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:29.546101 11298 webserver.cc:460] Webserver started at http://127.22.144.190:43775/ using document root <none> and password file <none>
I20260811 04:47:29.546676 11298 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:29.546742 11298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:29.546932 11298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:29.549100 11298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "068255ae0c434cc19d19fe52c9943f45"
format_stamp: "Formatted at 2026-08-11 04:47:29 on dist-test-slave-ss4n"
server_key: "1ef86cf7bcce73d0cbc2ad5389d74dd4"
server_key_iv: "46c310230cddb420f3d5c1c263a53650"
server_key_version: "encryptionkey@0"
I20260811 04:47:29.549746 11298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "068255ae0c434cc19d19fe52c9943f45"
format_stamp: "Formatted at 2026-08-11 04:47:29 on dist-test-slave-ss4n"
server_key: "1ef86cf7bcce73d0cbc2ad5389d74dd4"
server_key_iv: "46c310230cddb420f3d5c1c263a53650"
server_key_version: "encryptionkey@0"
I20260811 04:47:29.553802 11298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:29.556429 11317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:29.557893 11298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:29.558007 11298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "068255ae0c434cc19d19fe52c9943f45"
format_stamp: "Formatted at 2026-08-11 04:47:29 on dist-test-slave-ss4n"
server_key: "1ef86cf7bcce73d0cbc2ad5389d74dd4"
server_key_iv: "46c310230cddb420f3d5c1c263a53650"
server_key_version: "encryptionkey@0"
I20260811 04:47:29.558110 11298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:29.577752 11298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:29.581329 11298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:29.581576 11298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:29.589838 11298 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:33347
I20260811 04:47:29.589938 11378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:33347 every 8 connection(s)
I20260811 04:47:29.590968 11298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:29.593537 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 11298
I20260811 04:47:29.593703 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:29.593994 23106 external_mini_cluster.cc:1428] Setting key 34d246dd96e459fae1e88779a3fd67fe
I20260811 04:47:29.594365 11379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:29.602036 11379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45: Bootstrap starting.
Aug 11 04:47:29 dist-test-slave-ss4n krb5kdc[11284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:29.604656 11379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:29.605716 11379 log.cc:824] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:29.608275 11379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45: No bootstrap required, opened a new log
I20260811 04:47:29.609866 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:29.595515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.22.144.190:33347)
0811 04:47:29.595978 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:29.595998 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:29.596042 (+ 44us) server_negotiation.cc:407] Connection header received
0811 04:47:29.596839 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:29.596863 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:29.597315 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:29.597725 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:29.598442 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:29.599943 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:29.600851 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:29.601581 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:29.601761 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:29.604343 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:29.604350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:29.604367 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:29.604399 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:29.606559 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:29.607065 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:29.607071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:29.607078 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:29.607225 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:29.607504 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:29.607507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:29.607509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:29.608025 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:29.608220 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:29.608560 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 04:47:29.608879 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":129,"threads_started":1}
I20260811 04:47:29.612445 11379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "068255ae0c434cc19d19fe52c9943f45" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33347 } }
I20260811 04:47:29.612689 11379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:29.612736 11379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068255ae0c434cc19d19fe52c9943f45, State: Initialized, Role: FOLLOWER
I20260811 04:47:29.613111 11379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "068255ae0c434cc19d19fe52c9943f45" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33347 } }
I20260811 04:47:29.613225 11379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:29.613263 11379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:29.613335 11379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:29.614339 11379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "068255ae0c434cc19d19fe52c9943f45" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33347 } }
I20260811 04:47:29.614724 11379 leader_election.cc:302] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 068255ae0c434cc19d19fe52c9943f45; no voters:
I20260811 04:47:29.615085 11379 leader_election.cc:288] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:29.615309 11386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:29.615628 11386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [term 1 LEADER]: Becoming Leader. State: Replica: 068255ae0c434cc19d19fe52c9943f45, State: Running, Role: LEADER
I20260811 04:47:29.616010 11386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "068255ae0c434cc19d19fe52c9943f45" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33347 } }
I20260811 04:47:29.616478 11379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:29.617960 11389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 068255ae0c434cc19d19fe52c9943f45. Latest consensus state: current_term: 1 leader_uuid: "068255ae0c434cc19d19fe52c9943f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "068255ae0c434cc19d19fe52c9943f45" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33347 } } }
I20260811 04:47:29.617998 11387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "068255ae0c434cc19d19fe52c9943f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "068255ae0c434cc19d19fe52c9943f45" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33347 } } }
I20260811 04:47:29.618110 11389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:29.618114 11387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:29.618556 11399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:29.621857 11399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:29.628294 11399 catalog_manager.cc:1347] Generated new cluster ID: f0ce0d9271d543f4a77e532778004b67
I20260811 04:47:29.628402 11399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:29.639537 11399 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:29.640578 11399 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:29.651640 11399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45: Generated new TSK 0
I20260811 04:47:29.652529 11399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.144.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
Entry for principal kudu/127.22.144.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab.
I20260811 04:47:29.694217 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:33347
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:38609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:29.822399 11410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:29.822752 11410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:29.822847 11410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:29.827020 11410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:29.827114 11410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:29.827210 11410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:29.832609 11410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:38609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.129.keytab
--principal=kudu/127.22.144.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:29.833877 11410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:29.835009 11410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:29.845122 11417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:29.845122 11419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:29.845112 11416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:29.845542 11410 server_base.cc:1034] running on GCE node
I20260811 04:47:29.846050 11410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:29.846805 11410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:29.848006 11410 hybrid_clock.cc:648] HybridClock initialized: now 1786423649847991 us; error 27 us; skew 500 ppm
Aug 11 04:47:29 dist-test-slave-ss4n krb5kdc[11284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:29.850920 11410 init.cc:377] Logged in from keytab as kudu/127.22.144.129@KRBTEST.COM (short username kudu)
I20260811 04:47:29.851890 11410 webserver.cc:460] Webserver started at http://127.22.144.129:44845/ using document root <none> and password file <none>
I20260811 04:47:29.852803 11410 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:29.852917 11410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:29.853157 11410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:29.855316 11410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "fbea83a7538249c895078902c12e1824"
format_stamp: "Formatted at 2026-08-11 04:47:29 on dist-test-slave-ss4n"
server_key: "bc50ed4b79bbe1b1f965ceb959b5c0ee"
server_key_iv: "e3ae6f262ed219dd32b8b6590f442fc1"
server_key_version: "encryptionkey@0"
I20260811 04:47:29.855872 11410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "fbea83a7538249c895078902c12e1824"
format_stamp: "Formatted at 2026-08-11 04:47:29 on dist-test-slave-ss4n"
server_key: "bc50ed4b79bbe1b1f965ceb959b5c0ee"
server_key_iv: "e3ae6f262ed219dd32b8b6590f442fc1"
server_key_version: "encryptionkey@0"
I20260811 04:47:29.860003 11410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:29.862748 11427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:29.864045 11410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:29.864203 11410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "fbea83a7538249c895078902c12e1824"
format_stamp: "Formatted at 2026-08-11 04:47:29 on dist-test-slave-ss4n"
server_key: "bc50ed4b79bbe1b1f965ceb959b5c0ee"
server_key_iv: "e3ae6f262ed219dd32b8b6590f442fc1"
server_key_version: "encryptionkey@0"
I20260811 04:47:29.864336 11410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:29.880566 11410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:29.884308 11410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:29.884550 11410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:29.885236 11410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:29.886234 11410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:29.886317 11410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:29.886395 11410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:29.886440 11410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:29.896539 11410 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:40699
I20260811 04:47:29.896593 11563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:40699 every 8 connection(s)
I20260811 04:47:29.897596 11410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:47:29.903609 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 11410
I20260811 04:47:29.903779 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:47:29.904091 23106 external_mini_cluster.cc:1428] Setting key 967ac7615391cb9bd34fe493739feac4
Aug 11 04:47:29 dist-test-slave-ss4n krb5kdc[11284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.129@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:29.912829 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:29.899542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:54613 (local address 127.22.144.190:33347)
0811 04:47:29.899730 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:29.899736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:29.900599 (+ 863us) server_negotiation.cc:407] Connection header received
0811 04:47:29.901990 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:29.901993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:29.902043 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:29.902121 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:29.903388 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:29.904214 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:29.905743 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:29.906179 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:29.906278 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:29.909085 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:29.909088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:29.909090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:29.909114 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:29.910663 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:29.911219 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:29.911227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:29.911229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:29.911309 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:29.911670 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:29.911674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:29.911675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:29.911866 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:29.911978 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:29.912505 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 04:47:29.912660 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 04:47:29.913923 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:29.899969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:33347 (local address 127.22.144.129:54613)
0811 04:47:29.900434 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:29.900479 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:29.901790 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:29.902313 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:29.902320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:29.902715 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:29.903207 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:29.903221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:29.904351 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:29.904353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:29.905542 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:29.905558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:29.906515 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:29.906518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:29.906713 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:29.907365 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:29.907382 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:29.908906 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:29.910792 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:29.910799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:29.910816 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:29.911091 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:29.911407 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:29.911410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:29.911413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:29.911563 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:29.911978 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:29.911986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:29.912264 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 04:47:29.912461 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 04:47:29.912830 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.22.144.130@KRBTEST.COM; defaulting to no policy
I20260811 04:47:29.918706 11565 heartbeater.cc:347] Connected to a master server at 127.22.144.190:33347
I20260811 04:47:29.918983 11565 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:29.919667 11565 heartbeater.cc:511] Master 127.22.144.190:33347 requested a full tablet report, sending...
I20260811 04:47:29.921454 11336 ts_manager.cc:194] Registered new tserver with Master: fbea83a7538249c895078902c12e1824 (127.22.144.129:40699)
I20260811 04:47:29.923029 11336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.129@KRBTEST.COM'} at 127.22.144.129:54613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
Entry for principal kudu/127.22.144.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab.
I20260811 04:47:29.945842 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:33347
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:38609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:30.065086 11571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:30.065376 11571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:30.065421 11571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:30.069790 11571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:30.069864 11571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:30.069929 11571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:30.075455 11571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:38609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.130.keytab
--principal=kudu/127.22.144.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:30.076578 11571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:30.077731 11571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:30.087883 11576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:30.087883 11579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:30.087878 11577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:30.087925 11571 server_base.cc:1034] running on GCE node
I20260811 04:47:30.088600 11571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:30.089267 11571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:30.090435 11571 hybrid_clock.cc:648] HybridClock initialized: now 1786423650090413 us; error 64 us; skew 500 ppm
Aug 11 04:47:30 dist-test-slave-ss4n krb5kdc[11284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:30.093371 11571 init.cc:377] Logged in from keytab as kudu/127.22.144.130@KRBTEST.COM (short username kudu)
I20260811 04:47:30.094373 11571 webserver.cc:460] Webserver started at http://127.22.144.130:39661/ using document root <none> and password file <none>
I20260811 04:47:30.094960 11571 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:30.095024 11571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:30.095253 11571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:30.097400 11571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "c044638a8eaf4012a4594a1f2c9dde0a"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "8ad095efcfe681e1303e6cc4e07ae784"
server_key_iv: "5e9680a67180d7c37f50c50100df1275"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.098066 11571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "c044638a8eaf4012a4594a1f2c9dde0a"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "8ad095efcfe681e1303e6cc4e07ae784"
server_key_iv: "5e9680a67180d7c37f50c50100df1275"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.102066 11571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:47:30.104631 11587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.106060 11571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:30.106205 11571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "c044638a8eaf4012a4594a1f2c9dde0a"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "8ad095efcfe681e1303e6cc4e07ae784"
server_key_iv: "5e9680a67180d7c37f50c50100df1275"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.106328 11571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:30.133378 11571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:30.136946 11571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:30.137190 11571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:30.137930 11571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:30.138837 11571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:30.138916 11571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.138989 11571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:30.139045 11571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.148595 11719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:36541 every 8 connection(s)
I20260811 04:47:30.148595 11571 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:36541
I20260811 04:47:30.149745 11571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:30.154114 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 11571
I20260811 04:47:30.154270 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:47:30.154549 23106 external_mini_cluster.cc:1428] Setting key a0fabfc5e5ccabcb1a1446eeca50cdae
Aug 11 04:47:30 dist-test-slave-ss4n krb5kdc[11284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.130@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.131@KRBTEST.COM; defaulting to no policy
I20260811 04:47:30.165611 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.151740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:57679 (local address 127.22.144.190:33347)
0811 04:47:30.151893 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.151899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.152875 (+ 976us) server_negotiation.cc:407] Connection header received
0811 04:47:30.154375 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.154379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.154447 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.154508 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:30.156148 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.156903 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.158236 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.158457 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.158542 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.161668 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:30.161671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:30.161673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:30.161699 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:30.163361 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:30.163909 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:30.163912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:30.163914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:30.164001 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:30.164398 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:30.164401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:30.164403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:30.164606 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:30.164721 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.165260 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.165409 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:47:30.166467 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.152224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:33347 (local address 127.22.144.130:57679)
0811 04:47:30.152704 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.152751 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.154086 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.154911 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.154918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.155404 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:30.156000 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.156016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.157041 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.157044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.158074 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.158087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.158622 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.158628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.158857 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.159529 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:30.159556 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:30.161442 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:30.163495 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:30.163502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:30.163524 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:30.163792 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:30.164106 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:30.164110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:30.164112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:30.164287 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:30.164721 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:30.164729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:30.164997 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.165201 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.165592 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260811 04:47:30.167785 11720 heartbeater.cc:347] Connected to a master server at 127.22.144.190:33347
I20260811 04:47:30.168068 11720 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.131@KRBTEST.COM" created.
I20260811 04:47:30.168701 11720 heartbeater.cc:511] Master 127.22.144.190:33347 requested a full tablet report, sending...
I20260811 04:47:30.169956 11336 ts_manager.cc:194] Registered new tserver with Master: c044638a8eaf4012a4594a1f2c9dde0a (127.22.144.130:36541)
I20260811 04:47:30.170594 11336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.130@KRBTEST.COM'} at 127.22.144.130:57679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
Entry for principal kudu/127.22.144.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab.
I20260811 04:47:30.188508 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:33347
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:38609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:30.305096 11726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:30.305393 11726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:30.305437 11726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:30.309649 11726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:30.309725 11726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:30.309788 11726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:30.315459 11726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:38609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.131.keytab
--principal=kudu/127.22.144.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:33347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:30.316637 11726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:30.317795 11726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:30.328380 11734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:30.328366 11731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:30.328369 11732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:30.328435 11726 server_base.cc:1034] running on GCE node
I20260811 04:47:30.329069 11726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:30.329850 11726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:30.331070 11726 hybrid_clock.cc:648] HybridClock initialized: now 1786423650331036 us; error 58 us; skew 500 ppm
Aug 11 04:47:30 dist-test-slave-ss4n krb5kdc[11284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:30.334233 11726 init.cc:377] Logged in from keytab as kudu/127.22.144.131@KRBTEST.COM (short username kudu)
I20260811 04:47:30.335271 11726 webserver.cc:460] Webserver started at http://127.22.144.131:33687/ using document root <none> and password file <none>
I20260811 04:47:30.335872 11726 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:30.335935 11726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:30.336122 11726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:30.338320 11726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "16994d26fea1437b98082dda62d49ea4"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "07433977efbcd2a961c54b35ab8c21b5"
server_key_iv: "41d6e4196a0ce3c4d7706e29b3a193bd"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.338848 11726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "16994d26fea1437b98082dda62d49ea4"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "07433977efbcd2a961c54b35ab8c21b5"
server_key_iv: "41d6e4196a0ce3c4d7706e29b3a193bd"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.342834 11726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:30.345440 11743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.346969 11726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:30.347117 11726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "16994d26fea1437b98082dda62d49ea4"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "07433977efbcd2a961c54b35ab8c21b5"
server_key_iv: "41d6e4196a0ce3c4d7706e29b3a193bd"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.347272 11726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:30.361059 11726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:30.364530 11726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:30.364745 11726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:30.365374 11726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:30.366359 11726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:30.366408 11726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.366449 11726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:30.366504 11726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.376214 11726 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:32775
I20260811 04:47:30.376384 11873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:32775 every 8 connection(s)
I20260811 04:47:30.377223 11726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:30.386679 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 11726
I20260811 04:47:30.386832 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:47:30.387202 23106 external_mini_cluster.cc:1428] Setting key 2d69135dc596f8834bef611f81a60b9f
Aug 11 04:47:30 dist-test-slave-ss4n krb5kdc[11284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.131@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:30.392134 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.379231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:54163 (local address 127.22.144.190:33347)
0811 04:47:30.379374 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.379378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.380225 (+ 847us) server_negotiation.cc:407] Connection header received
0811 04:47:30.381715 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.381719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.381785 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.381891 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:30.383191 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.383747 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.384990 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.385225 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.385346 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.388405 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:30.388408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:30.388410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:30.388436 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:30.390049 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:30.390550 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:30.390553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:30.390554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:30.390637 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:30.391048 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:30.391051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:30.391053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:30.391256 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:30.391354 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.391878 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.391993 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:47:30.393110 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.379579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:33347 (local address 127.22.144.131:54163)
0811 04:47:30.380057 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.380103 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.381378 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.382095 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.382102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.382507 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:30.383024 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.383038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.383906 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.383909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.384848 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.384858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.385394 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.385398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.385652 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.386310 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:47:30.386327 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:30.388217 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:30.390169 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:30.390175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:30.390188 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:30.390455 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:30.390764 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:47:30.390766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:47:30.390768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:47:30.390921 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:47:30.391358 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:30.391364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:30.391623 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.391794 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.392078 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260811 04:47:30.394364 11874 heartbeater.cc:347] Connected to a master server at 127.22.144.190:33347
I20260811 04:47:30.394634 11874 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:30.395170 11874 heartbeater.cc:511] Master 127.22.144.190:33347 requested a full tablet report, sending...
I20260811 04:47:30.396311 11336 ts_manager.cc:194] Registered new tserver with Master: 16994d26fea1437b98082dda62d49ea4 (127.22.144.131:32775)
I20260811 04:47:30.396874 11336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.144.131@KRBTEST.COM'} at 127.22.144.131:54163
I20260811 04:47:30.401533 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:47:30.411090 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.403631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.22.144.190:33347)
0811 04:47:30.403796 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.403800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.403974 (+ 174us) server_negotiation.cc:407] Connection header received
0811 04:47:30.404069 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.404072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.404111 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.404204 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:30.404912 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.405433 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.406360 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.406586 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.406676 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.408094 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:30.408097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:30.408099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:30.408121 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:30.409450 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:30.409950 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:30.409953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:30.409954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:30.410033 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:30.410275 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:30.410277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:30.410278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:30.410488 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:30.410590 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.410867 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.410970 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 04:47:30.414108 11336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:47:30.417305 11336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:47:30.433303 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.427121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:32775 (local address 127.0.0.1:51566)
0811 04:47:30.427561 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.427609 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.427747 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.428385 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.428392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.428422 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:30.428616 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.428626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.430282 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.430286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.431558 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.431570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.432707 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.432711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.432818 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.432866 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.432979 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.433130 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":103,"thread_start_us":98,"threads_started":1}
I20260811 04:47:30.433302 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.427294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.22.144.131:32775)
0811 04:47:30.427628 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.427641 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.427688 (+ 47us) server_negotiation.cc:407] Connection header received
0811 04:47:30.427954 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.427964 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.428203 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.428784 (+ 581us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:30.428952 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.430128 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.431720 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.432575 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.432718 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.432864 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.433009 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.433116 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":144,"threads_started":1}
I20260811 04:47:30.433343 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.427804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:40699 (local address 127.0.0.1:46160)
0811 04:47:30.428075 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.428088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.428168 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.428806 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.428809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.428826 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:30.428949 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.428956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.430322 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.430325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.431632 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.431642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.432728 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.432731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.432851 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.432881 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.432957 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.433160 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":60,"thread_start_us":108,"threads_started":1}
I20260811 04:47:30.433915 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.428146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:36541 (local address 127.0.0.1:41176)
0811 04:47:30.428383 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.428395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.428464 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.429751 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.429754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.429774 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:30.429873 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.429879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.431122 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.431126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.432208 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.432219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.433514 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.433519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.433617 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.433644 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.433738 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.433803 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
I20260811 04:47:30.434284 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.427968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46160 (local address 127.22.144.129:40699)
0811 04:47:30.428346 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.428353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.428376 (+ 23us) server_negotiation.cc:407] Connection header received
0811 04:47:30.428452 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.428460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.428642 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.429087 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:30.429242 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.430185 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.431772 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.432587 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.433846 (+ 1259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.433998 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.434058 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.434166 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":86,"threads_started":1}
I20260811 04:47:30.434327 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.428736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.22.144.130:36541)
0811 04:47:30.429292 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.429303 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.429325 (+ 22us) server_negotiation.cc:407] Connection header received
0811 04:47:30.429395 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.429400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.429591 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.429752 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:30.429997 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.430966 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.432348 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.433173 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.433292 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.433416 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.434076 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.434178 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":132,"threads_started":1}
I20260811 04:47:30.436995 11641 tablet_service.cc:1467] Processing CreateTablet for tablet ba1dbde8f8e04e738a6ddd6bc80ba587 (DEFAULT_TABLE table=test-table [id=dc977ec228004e198c18875bf527a366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:30.436990 11800 tablet_service.cc:1467] Processing CreateTablet for tablet ba1dbde8f8e04e738a6ddd6bc80ba587 (DEFAULT_TABLE table=test-table [id=dc977ec228004e198c18875bf527a366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:30.438272 11800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1dbde8f8e04e738a6ddd6bc80ba587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:30.438272 11641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1dbde8f8e04e738a6ddd6bc80ba587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:30.439679 11484 tablet_service.cc:1467] Processing CreateTablet for tablet ba1dbde8f8e04e738a6ddd6bc80ba587 (DEFAULT_TABLE table=test-table [id=dc977ec228004e198c18875bf527a366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:30.440804 11484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1dbde8f8e04e738a6ddd6bc80ba587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:30.446317 11896 tablet_bootstrap.cc:492] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Bootstrap starting.
I20260811 04:47:30.447434 11895 tablet_bootstrap.cc:492] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Bootstrap starting.
I20260811 04:47:30.447649 11897 tablet_bootstrap.cc:492] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: Bootstrap starting.
I20260811 04:47:30.449993 11897 tablet_bootstrap.cc:654] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:30.450187 11895 tablet_bootstrap.cc:654] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:30.450999 11897 log.cc:824] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:30.451853 11895 log.cc:824] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:30.451993 11896 tablet_bootstrap.cc:654] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:30.453146 11896 log.cc:824] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:30.453322 11897 tablet_bootstrap.cc:492] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: No bootstrap required, opened a new log
I20260811 04:47:30.453562 11897 ts_tablet_manager.cc:1397] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:47:30.454011 11895 tablet_bootstrap.cc:492] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: No bootstrap required, opened a new log
I20260811 04:47:30.454202 11895 ts_tablet_manager.cc:1397] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 04:47:30.454991 11896 tablet_bootstrap.cc:492] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: No bootstrap required, opened a new log
I20260811 04:47:30.455204 11896 ts_tablet_manager.cc:1397] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260811 04:47:30.458173 11897 raft_consensus.cc:348] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.458484 11897 raft_consensus.cc:374] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:30.458370 11895 raft_consensus.cc:348] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.458552 11897 raft_consensus.cc:729] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbea83a7538249c895078902c12e1824, State: Initialized, Role: FOLLOWER
I20260811 04:47:30.458583 11895 raft_consensus.cc:374] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:30.458422 11896 raft_consensus.cc:348] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.458637 11895 raft_consensus.cc:729] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16994d26fea1437b98082dda62d49ea4, State: Initialized, Role: FOLLOWER
I20260811 04:47:30.458705 11896 raft_consensus.cc:374] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:30.458766 11896 raft_consensus.cc:729] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c044638a8eaf4012a4594a1f2c9dde0a, State: Initialized, Role: FOLLOWER
I20260811 04:47:30.459066 11895 consensus_queue.cc:260] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.459066 11897 consensus_queue.cc:260] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.459244 11896 consensus_queue.cc:260] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.459973 11720 heartbeater.cc:503] Master 127.22.144.190:33347 was elected leader, sending a full tablet report...
I20260811 04:47:30.460131 11895 ts_tablet_manager.cc:1428] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:47:30.460282 11896 ts_tablet_manager.cc:1428] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260811 04:47:30.460476 11874 heartbeater.cc:503] Master 127.22.144.190:33347 was elected leader, sending a full tablet report...
I20260811 04:47:30.460741 11897 ts_tablet_manager.cc:1428] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 04:47:30.461102 11565 heartbeater.cc:503] Master 127.22.144.190:33347 was elected leader, sending a full tablet report...
I20260811 04:47:30.605753 11906 raft_consensus.cc:482] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:30.605995 11906 raft_consensus.cc:504] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.607239 11906 leader_election.cc:288] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16994d26fea1437b98082dda62d49ea4 (127.22.144.131:32775), fbea83a7538249c895078902c12e1824 (127.22.144.129:40699)
I20260811 04:47:30.611362 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.607599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:41089 (local address 127.22.144.131:32775)
0811 04:47:30.607744 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.607749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.607852 (+ 103us) server_negotiation.cc:407] Connection header received
0811 04:47:30.608000 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.608006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.608077 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.608171 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:30.608486 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.609071 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.610243 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.610827 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.610957 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.611026 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.611203 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.611261 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 04:47:30.611399 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.607493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:32775 (local address 127.22.144.130:41089)
0811 04:47:30.607742 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.607757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.607879 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.608201 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.608204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.608225 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:30.608361 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.608370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.609189 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.609192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.610115 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.610125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.610935 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.610939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.611058 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.611085 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.611173 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.611243 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1}
I20260811 04:47:30.612089 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.607531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:40699 (local address 127.22.144.130:38343)
0811 04:47:30.607968 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.607982 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:30.608105 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:30.608436 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:30.608440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:30.608464 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:30.608593 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.608600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.609411 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.609414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.610523 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:30.610534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.611631 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.611635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:30.611771 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.611803 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 04:47:30.611886 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 04:47:30.611962 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":110,"threads_started":1}
I20260811 04:47:30.612207 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.607659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:38343 (local address 127.22.144.129:40699)
0811 04:47:30.607788 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.607794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.608086 (+ 292us) server_negotiation.cc:407] Connection header received
0811 04:47:30.608251 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.608254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.608310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.608400 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:30.608731 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.609305 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.610671 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.611358 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.611482 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.611532 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.611989 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.612081 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 04:47:30.612304 11821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dbde8f8e04e738a6ddd6bc80ba587" candidate_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16994d26fea1437b98082dda62d49ea4" is_pre_election: true
I20260811 04:47:30.612677 11821 raft_consensus.cc:2454] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c044638a8eaf4012a4594a1f2c9dde0a in term 0.
I20260811 04:47:30.612761 11509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dbde8f8e04e738a6ddd6bc80ba587" candidate_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbea83a7538249c895078902c12e1824" is_pre_election: true
I20260811 04:47:30.613068 11509 raft_consensus.cc:2454] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c044638a8eaf4012a4594a1f2c9dde0a in term 0.
I20260811 04:47:30.613255 11591 leader_election.cc:302] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16994d26fea1437b98082dda62d49ea4, c044638a8eaf4012a4594a1f2c9dde0a; no voters:
I20260811 04:47:30.613607 11906 raft_consensus.cc:2781] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:47:30.613713 11906 raft_consensus.cc:482] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:47:30.613758 11906 raft_consensus.cc:3037] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:30.614816 11906 raft_consensus.cc:504] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.615155 11906 leader_election.cc:288] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [CANDIDATE]: Term 1 election: Requested vote from peers 16994d26fea1437b98082dda62d49ea4 (127.22.144.131:32775), fbea83a7538249c895078902c12e1824 (127.22.144.129:40699)
I20260811 04:47:30.615531 11509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dbde8f8e04e738a6ddd6bc80ba587" candidate_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbea83a7538249c895078902c12e1824"
I20260811 04:47:30.615528 11821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1dbde8f8e04e738a6ddd6bc80ba587" candidate_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16994d26fea1437b98082dda62d49ea4"
I20260811 04:47:30.615672 11509 raft_consensus.cc:3037] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:30.615684 11821 raft_consensus.cc:3037] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:30.616946 11509 raft_consensus.cc:2454] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c044638a8eaf4012a4594a1f2c9dde0a in term 1.
I20260811 04:47:30.617129 11821 raft_consensus.cc:2454] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c044638a8eaf4012a4594a1f2c9dde0a in term 1.
I20260811 04:47:30.617291 11589 leader_election.cc:302] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c044638a8eaf4012a4594a1f2c9dde0a, fbea83a7538249c895078902c12e1824; no voters:
I20260811 04:47:30.617470 11906 raft_consensus.cc:2781] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:30.617796 11906 raft_consensus.cc:686] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 1 LEADER]: Becoming Leader. State: Replica: c044638a8eaf4012a4594a1f2c9dde0a, State: Running, Role: LEADER
I20260811 04:47:30.618217 11906 consensus_queue.cc:237] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } }
I20260811 04:47:30.621816 11334 catalog_manager.cc:5495] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a reported cstate change: term changed from 0 to 1, leader changed from <none> to c044638a8eaf4012a4594a1f2c9dde0a (127.22.144.130). New cstate: current_term: 1 leader_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c044638a8eaf4012a4594a1f2c9dde0a" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 36541 } health_report { overall_health: HEALTHY } } }
W20260811 04:47:30.628340 11875 tablet.cc:2369] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:47:30.649168 11566 tablet.cc:2369] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 04:47:30.650971 11721 tablet.cc:2369] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:47:30.675412 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.671597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.22.144.130:36541)
0811 04:47:30.671780 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.671785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.671799 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:47:30.671884 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.671887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.671944 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.672068 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:47:30.672339 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.672868 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.673820 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.674045 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.674161 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.674506 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:47:30.674959 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:47:30.675098 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.675165 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.675227 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 04:47:30.681725 11821 raft_consensus.cc:1261] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 1 FOLLOWER]: Refusing update from remote peer c044638a8eaf4012a4594a1f2c9dde0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:30.681725 11509 raft_consensus.cc:1261] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 1 FOLLOWER]: Refusing update from remote peer c044638a8eaf4012a4594a1f2c9dde0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:30.682488 11906 consensus_queue.cc:1035] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16994d26fea1437b98082dda62d49ea4" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:30.682749 11908 consensus_queue.cc:1035] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbea83a7538249c895078902c12e1824" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 40699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:30.689288 11911 mvcc.cc:205] Tried to move back new op lower bound from 7317191273184010240 to 7317191272934952960. Current Snapshot: MvccSnapshot[applied={T|T < 7317191273184010240}]
I20260811 04:47:30.691025 11912 mvcc.cc:205] Tried to move back new op lower bound from 7317191273184010240 to 7317191272934952960. Current Snapshot: MvccSnapshot[applied={T|T < 7317191273184010240}]
I20260811 04:47:30.701260 11334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:30.701426 11334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:30.704125 11334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 068255ae0c434cc19d19fe52c9943f45: Sending DeleteTablet for 3 replicas of tablet ba1dbde8f8e04e738a6ddd6bc80ba587
I20260811 04:47:30.705027 11484 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1dbde8f8e04e738a6ddd6bc80ba587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:30 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:46160
I20260811 04:47:30.705004 11800 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1dbde8f8e04e738a6ddd6bc80ba587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:30 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:51566
I20260811 04:47:30.705380 11641 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1dbde8f8e04e738a6ddd6bc80ba587 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:30 UTC) from {username='kudu', principal='kudu/127.22.144.190@KRBTEST.COM'} at 127.0.0.1:41176
I20260811 04:47:30.705610 11921 tablet_replica.cc:330] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: stopping tablet replica
I20260811 04:47:30.705902 11923 tablet_replica.cc:330] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: stopping tablet replica
I20260811 04:47:30.706022 11921 raft_consensus.cc:2229] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:30.706382 11921 raft_consensus.cc:2258] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:30.706542 11922 tablet_replica.cc:330] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824: stopping tablet replica
I20260811 04:47:30.706851 11922 raft_consensus.cc:2229] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:30.706883 11923 raft_consensus.cc:2229] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:47:30.706949 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 11410
I20260811 04:47:30.707244 11922 raft_consensus.cc:2258] T ba1dbde8f8e04e738a6ddd6bc80ba587 P fbea83a7538249c895078902c12e1824 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:30.715097 11921 ts_tablet_manager.cc:1905] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:30.716677 11923 raft_consensus.cc:2258] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:30.718501 11921 ts_tablet_manager.cc:1918] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:30.718662 11921 log.cc:1196] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/ba1dbde8f8e04e738a6ddd6bc80ba587
I20260811 04:47:30.719020 11921 ts_tablet_manager.cc:1939] T ba1dbde8f8e04e738a6ddd6bc80ba587 P 16994d26fea1437b98082dda62d49ea4: Deleting consensus metadata
I20260811 04:47:30.720389 11321 catalog_manager.cc:4841] TS 16994d26fea1437b98082dda62d49ea4 (127.22.144.131:32775): tablet ba1dbde8f8e04e738a6ddd6bc80ba587 (table test-table [id=dc977ec228004e198c18875bf527a366]) successfully deleted
W20260811 04:47:30.720945 11319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:40699 (error 108)
I20260811 04:47:30.721467 11923 ts_tablet_manager.cc:1905] T ba1dbde8f8e04e738a6ddd6bc80ba587 P c044638a8eaf4012a4594a1f2c9dde0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:30.721966 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 11571
I20260811 04:47:30.725972 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.725519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:40699 (local address 127.0.0.1:46172)
0811 04:47:30.725766 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.725853 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:40699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260811 04:47:30.726171 11319 catalog_manager.cc:4584] TS fbea83a7538249c895078902c12e1824 (127.22.144.129:40699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba1dbde8f8e04e738a6ddd6bc80ba587: Network error: Client connection negotiation failed: client connection to 127.22.144.129:40699: connect: Connection refused (error 111)
I20260811 04:47:30.731596 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 11726
I20260811 04:47:30.734709 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.731735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:36541 (local address 127.0.0.1:41186)
0811 04:47:30.734511 (+ 2776us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:30.734590 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.130:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2654}
I20260811 04:47:30.739959 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 11298
2026-08-11T04:47:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2640 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T04:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:30Z Disabled control of system clock
I20260811 04:47:30.776083 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:45441
--webserver_interface=127.22.144.190
--webserver_port=0
--builtin_ntp_servers=127.22.144.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:45441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 04:47:30.892370 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:30.892650 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:30.892690 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:30.896838 11928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:30.896914 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:30.896929 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:30.896946 11928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:30.896961 11928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:30.901968 11928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:45441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:45441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:30.903179 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:30.904608 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:30.913997 11934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:30.913987 11936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:30.913987 11933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:30.914237 11928 server_base.cc:1034] running on GCE node
I20260811 04:47:30.914973 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:30.916131 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:30.917317 11928 hybrid_clock.cc:648] HybridClock initialized: now 1786423650917285 us; error 54 us; skew 500 ppm
I20260811 04:47:30.919528 11928 webserver.cc:460] Webserver started at http://127.22.144.190:34061/ using document root <none> and password file <none>
I20260811 04:47:30.920176 11928 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:30.920275 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:30.920524 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:30.923079 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "4b2ca20933eaea46341b1327964fe962"
server_key_iv: "fc0d554000f68205ff2ce758e1634873"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.923718 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "4b2ca20933eaea46341b1327964fe962"
server_key_iv: "fc0d554000f68205ff2ce758e1634873"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.927698 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:30.930326 11942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:30.931620 11928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:47:30.931756 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9"
format_stamp: "Formatted at 2026-08-11 04:47:30 on dist-test-slave-ss4n"
server_key: "4b2ca20933eaea46341b1327964fe962"
server_key_iv: "fc0d554000f68205ff2ce758e1634873"
server_key_version: "encryptionkey@0"
I20260811 04:47:30.931880 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:30.946910 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:30.947700 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:30.947916 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:30.955827 11928 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:45441
I20260811 04:47:30.955842 12002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:45441 every 8 connection(s)
I20260811 04:47:30.956915 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:30.960245 12003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:30.963194 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 11928
I20260811 04:47:30.963333 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:30.963672 23106 external_mini_cluster.cc:1428] Setting key 6106882319c0c06c1e31390dbc65c348
I20260811 04:47:30.967038 12003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: Bootstrap starting.
I20260811 04:47:30.969329 12003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:30.970145 12003 log.cc:824] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:30.972535 12003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: No bootstrap required, opened a new log
I20260811 04:47:30.973803 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:30.965284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.22.144.190:45441)
0811 04:47:30.966014 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:30.966029 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:30.966065 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:47:30.966696 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:30.966714 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:30.967170 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:30.967550 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:30.968020 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.969022 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.970166 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:30.970826 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:30.971013 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:30.971909 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:30.971914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:30.971928 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:30.972334 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:30.972389 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:30.972410 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:30.972490 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:30.972705 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 04:47:30.973009 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":170,"threads_started":1}
I20260811 04:47:30.976661 12003 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45441 } }
I20260811 04:47:30.976879 12003 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:30.976955 12003 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1c1e8b1599473eb02ae7e37c2fcce9, State: Initialized, Role: FOLLOWER
I20260811 04:47:30.977420 12003 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45441 } }
I20260811 04:47:30.977627 12003 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:30.977698 12003 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:30.977821 12003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:30.978801 12003 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45441 } }
I20260811 04:47:30.979164 12003 leader_election.cc:302] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1c1e8b1599473eb02ae7e37c2fcce9; no voters:
I20260811 04:47:30.979490 12003 leader_election.cc:288] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:30.979624 12008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:30.979907 12008 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [term 1 LEADER]: Becoming Leader. State: Replica: ad1c1e8b1599473eb02ae7e37c2fcce9, State: Running, Role: LEADER
I20260811 04:47:30.980216 12008 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45441 } }
I20260811 04:47:30.980783 12003 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:30.982422 12010 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1c1e8b1599473eb02ae7e37c2fcce9. Latest consensus state: current_term: 1 leader_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45441 } } }
I20260811 04:47:30.982546 12010 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:30.982360 12009 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1c1e8b1599473eb02ae7e37c2fcce9" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 45441 } } }
I20260811 04:47:30.982640 12009 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9 [sys.catalog]: This master's current role is: LEADER
W20260811 04:47:30.986711 12025 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:47:30.986806 12025 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:47:30.986878 12019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:30.987553 12019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:30.993521 12019 catalog_manager.cc:1347] Generated new cluster ID: 2ab00f4d2a5f4842ae8dc57ff49a7b6d
I20260811 04:47:30.993620 12019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:31.007885 12019 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:31.008847 12019 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:31.014660 12019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: Generated new TSK 0
I20260811 04:47:31.015476 12019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:31.019810 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.129:0
--local_ip_for_outbound_sockets=127.22.144.129
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45441
--builtin_ntp_servers=127.22.144.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 04:47:31.139714 12029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:31.139998 12029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:31.140038 12029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:31.144094 12029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:31.144147 12029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:31.144204 12029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.129
I20260811 04:47:31.148929 12029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.144.129
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.129
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:31.150069 12029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:31.151909 12029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:47:31.162137 12029 server_base.cc:1034] running on GCE node
W20260811 04:47:31.162092 12038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:31.162077 12035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:31.162081 12040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:31.162953 12029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:31.163609 12029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:31.164803 12029 hybrid_clock.cc:648] HybridClock initialized: now 1786423651164785 us; error 30 us; skew 500 ppm
I20260811 04:47:31.166724 12029 webserver.cc:460] Webserver started at http://127.22.144.129:44947/ using document root <none> and password file <none>
I20260811 04:47:31.167346 12029 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:31.167441 12029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:31.167691 12029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:31.169850 12029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data/instance:
uuid: "978ba619a8ed427881f56945940d0c96"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "13ec62cde82bda3d151136ba8776edb5"
server_key_iv: "3da558f3de6a2eb0c4b98d9579f9c121"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.170454 12029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance:
uuid: "978ba619a8ed427881f56945940d0c96"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "13ec62cde82bda3d151136ba8776edb5"
server_key_iv: "3da558f3de6a2eb0c4b98d9579f9c121"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.174268 12029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 04:47:31.176725 12048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.178001 12029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 04:47:31.178164 12029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal
uuid: "978ba619a8ed427881f56945940d0c96"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "13ec62cde82bda3d151136ba8776edb5"
server_key_iv: "3da558f3de6a2eb0c4b98d9579f9c121"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.178285 12029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:31.191448 12029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:31.192122 12029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:31.192332 12029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:31.192921 12029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:31.193904 12029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:31.193976 12029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.194048 12029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:31.194096 12029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.203218 12029 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.129:37713
I20260811 04:47:31.203250 12176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.129:37713 every 8 connection(s)
I20260811 04:47:31.204125 12029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/data/info.pb
I20260811 04:47:31.207743 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12029
I20260811 04:47:31.207887 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-0/wal/instance
I20260811 04:47:31.208200 23106 external_mini_cluster.cc:1428] Setting key 39c648e7c201f0173f3b1c90ad5cc79f
I20260811 04:47:31.210472 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.130:0
--local_ip_for_outbound_sockets=127.22.144.130
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45441
--builtin_ntp_servers=127.22.144.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 04:47:31.215191 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.206054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:42053 (local address 127.22.144.190:45441)
0811 04:47:31.206204 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.206208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.207287 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 04:47:31.209222 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.209225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.209281 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.209390 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:31.210631 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.211202 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.212846 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.213223 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.213348 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.214564 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:31.214570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:31.214577 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:31.214596 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:31.214621 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:31.214634 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:31.214725 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.214881 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.214962 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:47:31.216032 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.206452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:45441 (local address 127.22.144.129:42053)
0811 04:47:31.207129 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.207178 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.208961 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.209608 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.209615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.209708 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:31.210447 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.210469 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.211353 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.211359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.212714 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.212726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.213380 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.213383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.213609 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.214280 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:31.214282 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:31.214324 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:31.214392 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:31.214732 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:31.214740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:31.214744 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.214842 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.215080 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":130,"threads_started":1}
I20260811 04:47:31.217547 12177 heartbeater.cc:347] Connected to a master server at 127.22.144.190:45441
I20260811 04:47:31.217801 12177 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:31.218350 12177 heartbeater.cc:511] Master 127.22.144.190:45441 requested a full tablet report, sending...
I20260811 04:47:31.220180 11961 ts_manager.cc:194] Registered new tserver with Master: 978ba619a8ed427881f56945940d0c96 (127.22.144.129:37713)
I20260811 04:47:31.222026 11961 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.129:42053
W20260811 04:47:31.328089 12180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:31.328387 12180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:31.328428 12180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:31.332695 12180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:31.332767 12180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:31.332824 12180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.130
I20260811 04:47:31.337853 12180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.144.130
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.130
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:31.338979 12180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:31.340371 12180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:31.350353 12190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:31.350335 12191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:31.350328 12193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:31.350800 12180 server_base.cc:1034] running on GCE node
I20260811 04:47:31.351253 12180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:31.351926 12180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:31.353094 12180 hybrid_clock.cc:648] HybridClock initialized: now 1786423651353073 us; error 32 us; skew 500 ppm
I20260811 04:47:31.355046 12180 webserver.cc:460] Webserver started at http://127.22.144.130:46323/ using document root <none> and password file <none>
I20260811 04:47:31.355639 12180 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:31.355712 12180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:31.355976 12180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:31.358354 12180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data/instance:
uuid: "37378dada7cf4c97a05e7550b3c5975e"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "84fee88405d9a28148d87c1ae7c14dc2"
server_key_iv: "214483c91fd3e4921d22e136d347da78"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.358978 12180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance:
uuid: "37378dada7cf4c97a05e7550b3c5975e"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "84fee88405d9a28148d87c1ae7c14dc2"
server_key_iv: "214483c91fd3e4921d22e136d347da78"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.362927 12180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:31.365460 12203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.366870 12180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:31.367014 12180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal
uuid: "37378dada7cf4c97a05e7550b3c5975e"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "84fee88405d9a28148d87c1ae7c14dc2"
server_key_iv: "214483c91fd3e4921d22e136d347da78"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.367139 12180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:31.382800 12180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:31.383566 12180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:31.383787 12180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:31.384405 12180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:31.385375 12180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:31.385449 12180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.385545 12180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:31.385592 12180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.395350 12180 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.130:35531
I20260811 04:47:31.395371 12332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.130:35531 every 8 connection(s)
I20260811 04:47:31.396385 12180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/data/info.pb
I20260811 04:47:31.398802 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12180
I20260811 04:47:31.398959 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal/instance
I20260811 04:47:31.399267 23106 external_mini_cluster.cc:1428] Setting key aed4c2ae2ff388ab62f25630cdeb67e8
I20260811 04:47:31.401926 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.144.131:0
--local_ip_for_outbound_sockets=127.22.144.131
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45441
--builtin_ntp_servers=127.22.144.148:33747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 04:47:31.407423 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.398638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.130:47893 (local address 127.22.144.190:45441)
0811 04:47:31.398825 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.398831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.400218 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 04:47:31.401970 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.401973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.402055 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.402185 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:31.403245 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.403802 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.405145 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.405378 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.405467 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.406920 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:31.406923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:31.406925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:31.406938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:31.406957 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:31.406965 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:31.407027 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.407251 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.407306 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 04:47:31.408810 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.399181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:45441 (local address 127.22.144.130:47893)
0811 04:47:31.400008 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.400066 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.401706 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.402475 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.402483 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.402574 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:31.403083 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.403097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.403944 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.403948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.404998 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.405023 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.405575 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.405581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.405841 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.406580 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:31.406584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:31.406654 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:31.406754 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:31.407061 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:31.407074 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:31.407083 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.407248 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.407615 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":151,"threads_started":1}
I20260811 04:47:31.410233 12333 heartbeater.cc:347] Connected to a master server at 127.22.144.190:45441
I20260811 04:47:31.410519 12333 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:31.411098 12333 heartbeater.cc:511] Master 127.22.144.190:45441 requested a full tablet report, sending...
I20260811 04:47:31.412243 11961 ts_manager.cc:194] Registered new tserver with Master: 37378dada7cf4c97a05e7550b3c5975e (127.22.144.130:35531)
I20260811 04:47:31.412829 11961 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.130:47893
W20260811 04:47:31.523797 12338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:31.524122 12338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:31.524204 12338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:31.528383 12338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:31.528483 12338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:31.528579 12338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.144.131
I20260811 04:47:31.533617 12338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:33747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.144.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.144.131
--webserver_port=0
--tserver_master_addrs=127.22.144.190:45441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.144.131
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:31.534821 12338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:31.536370 12338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:31.546624 12347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:31.546711 12345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:31.546859 12338 server_base.cc:1034] running on GCE node
W20260811 04:47:31.546622 12344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:31.547576 12338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:31.548293 12338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:31.549530 12338 hybrid_clock.cc:648] HybridClock initialized: now 1786423651549472 us; error 28 us; skew 500 ppm
I20260811 04:47:31.551441 12338 webserver.cc:460] Webserver started at http://127.22.144.131:43853/ using document root <none> and password file <none>
I20260811 04:47:31.552047 12338 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:31.552145 12338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:31.552394 12338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:31.554765 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data/instance:
uuid: "c72209c0d8de44a0b6d16427eeae0aec"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "f5264402b861fa112884dfb901cb5eb4"
server_key_iv: "4666dd04436e9f872eccd5595b9d9d80"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.555420 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance:
uuid: "c72209c0d8de44a0b6d16427eeae0aec"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "f5264402b861fa112884dfb901cb5eb4"
server_key_iv: "4666dd04436e9f872eccd5595b9d9d80"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.559499 12338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:31.562096 12355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.563414 12338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:47:31.563563 12338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal
uuid: "c72209c0d8de44a0b6d16427eeae0aec"
format_stamp: "Formatted at 2026-08-11 04:47:31 on dist-test-slave-ss4n"
server_key: "f5264402b861fa112884dfb901cb5eb4"
server_key_iv: "4666dd04436e9f872eccd5595b9d9d80"
server_key_version: "encryptionkey@0"
I20260811 04:47:31.563694 12338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:31.589202 12338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:31.590013 12338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:31.590240 12338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:31.590881 12338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 04:47:31.591931 12338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 04:47:31.592006 12338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.592082 12338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 04:47:31.592137 12338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:31.601830 12338 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.131:44195
I20260811 04:47:31.601889 12487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.131:44195 every 8 connection(s)
I20260811 04:47:31.602843 12338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/data/info.pb
I20260811 04:47:31.610545 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12338
I20260811 04:47:31.610684 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal/instance
I20260811 04:47:31.610999 23106 external_mini_cluster.cc:1428] Setting key df0c6e28924bd03b02aef5932be1749e
I20260811 04:47:31.612473 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.604748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.131:33297 (local address 127.22.144.190:45441)
0811 04:47:31.604906 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.604911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.606118 (+ 1207us) server_negotiation.cc:407] Connection header received
0811 04:47:31.607474 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.607477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.607527 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.607605 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:31.608504 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.609036 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.610234 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.610454 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.610536 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.611964 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:31.611967 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:31.611969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:31.611980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:31.611999 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:31.612006 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:31.612090 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.612262 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.612329 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:47:31.613538 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.605230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:45441 (local address 127.22.144.131:33297)
0811 04:47:31.605953 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.605994 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.607273 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.607790 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.607797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.607881 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:47:31.608374 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.608386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.609150 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.609153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.610121 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.610131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.610633 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.610639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.610877 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.611598 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 04:47:31.611600 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:47:31.611643 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 04:47:31.611752 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:47:31.612099 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:47:31.612111 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:47:31.612117 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.612261 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.612572 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":123,"threads_started":1}
I20260811 04:47:31.614596 12488 heartbeater.cc:347] Connected to a master server at 127.22.144.190:45441
I20260811 04:47:31.614861 12488 heartbeater.cc:464] Registering TS with master...
I20260811 04:47:31.615401 12488 heartbeater.cc:511] Master 127.22.144.190:45441 requested a full tablet report, sending...
I20260811 04:47:31.616418 11961 ts_manager.cc:194] Registered new tserver with Master: c72209c0d8de44a0b6d16427eeae0aec (127.22.144.131:44195)
I20260811 04:47:31.617008 11961 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.144.131:33297
I20260811 04:47:31.625219 23106 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 04:47:31.631399 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.627358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.22.144.190:45441)
0811 04:47:31.627529 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.627534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.627692 (+ 158us) server_negotiation.cc:407] Connection header received
0811 04:47:31.627838 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.627842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.627909 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.627997 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:31.628294 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.628826 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.629780 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.630055 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.630174 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.630964 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:31.630968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:31.630971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:31.630985 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:31.631012 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:31.631020 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:31.631097 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.631196 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.631252 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 04:47:31.634482 11961 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 04:47:31.637781 11961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 04:47:31.653175 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.646956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:37713 (local address 127.0.0.1:46714)
0811 04:47:31.647365 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.647420 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.648402 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.648949 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.648953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.648970 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:31.649063 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.649071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.650228 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.650233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.651729 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.651740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.652686 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.652689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.652816 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.652852 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.652938 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.653028 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":216,"spinlock_wait_cycles":4864,"thread_start_us":96,"threads_started":1}
I20260811 04:47:31.653271 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.647102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.22.144.129:37713)
0811 04:47:31.647599 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.647618 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.647647 (+ 29us) server_negotiation.cc:407] Connection header received
0811 04:47:31.648575 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.648582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.648766 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.648950 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:31.649195 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.650109 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.651861 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.652549 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.652675 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.652905 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.652970 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.653076 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":141,"threads_started":1}
I20260811 04:47:31.653959 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.647726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:35531 (local address 127.0.0.1:46690)
0811 04:47:31.648295 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.648307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.648402 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.649142 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.649145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.649164 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:31.649241 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.649248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.650603 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.650607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.652120 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.652128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.653392 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.653395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.653529 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.653558 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.653625 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.653712 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":58,"spinlock_wait_cycles":6784,"thread_start_us":94,"threads_started":1}
I20260811 04:47:31.654896 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.647360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44195 (local address 127.0.0.1:55448)
0811 04:47:31.647888 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.647927 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.648077 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.648751 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.648757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.648784 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:31.648924 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.648931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.650467 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.650470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.652116 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.652128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.654498 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.654501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.654599 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.654617 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.654709 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.654779 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":393,"thread_start_us":93,"threads_started":1}
I20260811 04:47:31.655009 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.647817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.22.144.131:44195)
0811 04:47:31.648208 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.648222 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.648245 (+ 23us) server_negotiation.cc:407] Connection header received
0811 04:47:31.648324 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.648334 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.648571 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.648727 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:31.649076 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.650329 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.653575 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.654388 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.654513 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.654676 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.654756 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.654848 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":105,"threads_started":1}
I20260811 04:47:31.655459 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.648294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.22.144.130:35531)
0811 04:47:31.648713 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.648726 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.648747 (+ 21us) server_negotiation.cc:407] Connection header received
0811 04:47:31.648816 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.648821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.649010 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.649169 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:31.649359 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.650487 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.652257 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.653147 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.653309 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.655199 (+ 1890us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.655250 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.655340 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":126,"threads_started":1}
I20260811 04:47:31.657076 12102 tablet_service.cc:1467] Processing CreateTablet for tablet 40b4de0f163c4b58ba2bfa0b392e476f (DEFAULT_TABLE table=test-table [id=822a3d3e753443a3a3f98c32d0ec22ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:31.657405 12409 tablet_service.cc:1467] Processing CreateTablet for tablet 40b4de0f163c4b58ba2bfa0b392e476f (DEFAULT_TABLE table=test-table [id=822a3d3e753443a3a3f98c32d0ec22ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:31.658521 12102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b4de0f163c4b58ba2bfa0b392e476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:31.658531 12409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b4de0f163c4b58ba2bfa0b392e476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:31.658164 12258 tablet_service.cc:1467] Processing CreateTablet for tablet 40b4de0f163c4b58ba2bfa0b392e476f (DEFAULT_TABLE table=test-table [id=822a3d3e753443a3a3f98c32d0ec22ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 04:47:31.659099 12258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b4de0f163c4b58ba2bfa0b392e476f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:31.664721 12508 tablet_bootstrap.cc:492] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Bootstrap starting.
I20260811 04:47:31.666224 12510 tablet_bootstrap.cc:492] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Bootstrap starting.
I20260811 04:47:31.667179 12508 tablet_bootstrap.cc:654] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:31.668042 12508 log.cc:824] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:31.668139 12510 tablet_bootstrap.cc:654] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:31.668874 12509 tablet_bootstrap.cc:492] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: Bootstrap starting.
I20260811 04:47:31.668999 12510 log.cc:824] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:31.670780 12509 tablet_bootstrap.cc:654] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:31.670984 12510 tablet_bootstrap.cc:492] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: No bootstrap required, opened a new log
I20260811 04:47:31.671202 12510 ts_tablet_manager.cc:1397] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 04:47:31.671660 12509 log.cc:824] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:31.673210 12508 tablet_bootstrap.cc:492] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: No bootstrap required, opened a new log
I20260811 04:47:31.673427 12508 ts_tablet_manager.cc:1397] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260811 04:47:31.673725 12509 tablet_bootstrap.cc:492] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: No bootstrap required, opened a new log
I20260811 04:47:31.673910 12509 ts_tablet_manager.cc:1397] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 04:47:31.675652 12510 raft_consensus.cc:348] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.675984 12510 raft_consensus.cc:374] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:31.676047 12510 raft_consensus.cc:729] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37378dada7cf4c97a05e7550b3c5975e, State: Initialized, Role: FOLLOWER
I20260811 04:47:31.676752 12510 consensus_queue.cc:260] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.677469 12333 heartbeater.cc:503] Master 127.22.144.190:45441 was elected leader, sending a full tablet report...
I20260811 04:47:31.677891 12510 ts_tablet_manager.cc:1428] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 04:47:31.678519 12508 raft_consensus.cc:348] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.678607 12509 raft_consensus.cc:348] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.678807 12508 raft_consensus.cc:374] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:31.678872 12508 raft_consensus.cc:729] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c72209c0d8de44a0b6d16427eeae0aec, State: Initialized, Role: FOLLOWER
I20260811 04:47:31.678894 12509 raft_consensus.cc:374] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:31.678966 12509 raft_consensus.cc:729] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978ba619a8ed427881f56945940d0c96, State: Initialized, Role: FOLLOWER
I20260811 04:47:31.679337 12508 consensus_queue.cc:260] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.679392 12509 consensus_queue.cc:260] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.680203 12177 heartbeater.cc:503] Master 127.22.144.190:45441 was elected leader, sending a full tablet report...
I20260811 04:47:31.680390 12509 ts_tablet_manager.cc:1428] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 04:47:31.680426 12508 ts_tablet_manager.cc:1428] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 04:47:31.680663 12488 heartbeater.cc:503] Master 127.22.144.190:45441 was elected leader, sending a full tablet report...
I20260811 04:47:31.698951 12518 raft_consensus.cc:482] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 04:47:31.699146 12518 raft_consensus.cc:504] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.700398 12518 leader_election.cc:288] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37378dada7cf4c97a05e7550b3c5975e (127.22.144.130:35531), c72209c0d8de44a0b6d16427eeae0aec (127.22.144.131:44195)
I20260811 04:47:31.704631 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.700923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:60361 (local address 127.22.144.131:44195)
0811 04:47:31.701072 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.701079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.701103 (+ 24us) server_negotiation.cc:407] Connection header received
0811 04:47:31.701269 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.701274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.701334 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.701399 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:31.701808 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.702347 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.703488 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.703889 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.703991 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.704031 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.704411 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.704492 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 04:47:31.704631 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.700765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.144.129:40207 (local address 127.22.144.130:35531)
0811 04:47:31.700917 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.700922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.701203 (+ 281us) server_negotiation.cc:407] Connection header received
0811 04:47:31.701340 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.701343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.701414 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.701521 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 04:47:31.701877 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.702379 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.703547 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.703923 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.704164 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.704217 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.704409 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.704492 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 04:47:31.704632 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.700656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.131:44195 (local address 127.22.144.129:60361)
0811 04:47:31.701021 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.701034 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.701114 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.701436 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.701441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.701463 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:31.701676 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.701685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.702467 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.702473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.703352 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.703361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.704118 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.704123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.704275 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.704294 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.704391 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.704513 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":119}
I20260811 04:47:31.704650 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.700678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.130:35531 (local address 127.22.144.129:40207)
0811 04:47:31.701069 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.701083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:47:31.701229 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:47:31.701547 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:47:31.701549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:47:31.701564 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 04:47:31.701720 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.701726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.702521 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.702523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.703378 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:47:31.703387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.704171 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.704174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:47:31.704270 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.704291 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 04:47:31.704385 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 04:47:31.704492 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":58,"thread_start_us":139,"threads_started":1}
W20260811 04:47:31.705423 12178 tablet.cc:2369] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 04:47:31.705446 12433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b4de0f163c4b58ba2bfa0b392e476f" candidate_uuid: "978ba619a8ed427881f56945940d0c96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c72209c0d8de44a0b6d16427eeae0aec" is_pre_election: true
I20260811 04:47:31.705651 12279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b4de0f163c4b58ba2bfa0b392e476f" candidate_uuid: "978ba619a8ed427881f56945940d0c96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37378dada7cf4c97a05e7550b3c5975e" is_pre_election: true
I20260811 04:47:31.705839 12433 raft_consensus.cc:2454] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978ba619a8ed427881f56945940d0c96 in term 0.
I20260811 04:47:31.705896 12279 raft_consensus.cc:2454] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978ba619a8ed427881f56945940d0c96 in term 0.
I20260811 04:47:31.706411 12051 leader_election.cc:302] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 37378dada7cf4c97a05e7550b3c5975e, 978ba619a8ed427881f56945940d0c96, c72209c0d8de44a0b6d16427eeae0aec; no voters:
I20260811 04:47:31.706743 12518 raft_consensus.cc:2781] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 04:47:31.706833 12518 raft_consensus.cc:482] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 04:47:31.706877 12518 raft_consensus.cc:3037] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:31.708069 12518 raft_consensus.cc:504] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.708516 12518 leader_election.cc:288] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [CANDIDATE]: Term 1 election: Requested vote from peers 37378dada7cf4c97a05e7550b3c5975e (127.22.144.130:35531), c72209c0d8de44a0b6d16427eeae0aec (127.22.144.131:44195)
I20260811 04:47:31.708899 12279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b4de0f163c4b58ba2bfa0b392e476f" candidate_uuid: "978ba619a8ed427881f56945940d0c96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37378dada7cf4c97a05e7550b3c5975e"
I20260811 04:47:31.708901 12433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b4de0f163c4b58ba2bfa0b392e476f" candidate_uuid: "978ba619a8ed427881f56945940d0c96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c72209c0d8de44a0b6d16427eeae0aec"
I20260811 04:47:31.709020 12279 raft_consensus.cc:3037] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:31.709048 12433 raft_consensus.cc:3037] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:31.710247 12433 raft_consensus.cc:2454] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978ba619a8ed427881f56945940d0c96 in term 1.
I20260811 04:47:31.710253 12279 raft_consensus.cc:2454] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978ba619a8ed427881f56945940d0c96 in term 1.
I20260811 04:47:31.710609 12051 leader_election.cc:302] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37378dada7cf4c97a05e7550b3c5975e, 978ba619a8ed427881f56945940d0c96; no voters:
I20260811 04:47:31.710942 12518 raft_consensus.cc:2781] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:31.711218 12518 raft_consensus.cc:686] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 1 LEADER]: Becoming Leader. State: Replica: 978ba619a8ed427881f56945940d0c96, State: Running, Role: LEADER
I20260811 04:47:31.711581 12518 consensus_queue.cc:237] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } }
I20260811 04:47:31.715018 11959 catalog_manager.cc:5495] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 978ba619a8ed427881f56945940d0c96 (127.22.144.129). New cstate: current_term: 1 leader_uuid: "978ba619a8ed427881f56945940d0c96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978ba619a8ed427881f56945940d0c96" member_type: VOTER last_known_addr { host: "127.22.144.129" port: 37713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 } health_report { overall_health: UNKNOWN } } }
I20260811 04:47:31.732374 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.729092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.22.144.129:37713)
0811 04:47:31.729227 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:31.729231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:31.729243 (+ 12us) server_negotiation.cc:407] Connection header received
0811 04:47:31.729322 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:31.729324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:31.729382 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:31.729471 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:47:31.729766 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.730274 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.731092 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:31.731304 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:31.731377 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:31.731634 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:47:31.732065 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:47:31.732157 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:31.732215 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 04:47:31.732261 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 04:47:31.738548 12279 raft_consensus.cc:1261] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 1 FOLLOWER]: Refusing update from remote peer 978ba619a8ed427881f56945940d0c96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:31.738548 12433 raft_consensus.cc:1261] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 1 FOLLOWER]: Refusing update from remote peer 978ba619a8ed427881f56945940d0c96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:47:31.739322 12520 consensus_queue.cc:1035] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37378dada7cf4c97a05e7550b3c5975e" member_type: VOTER last_known_addr { host: "127.22.144.130" port: 35531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:31.739544 12518 consensus_queue.cc:1035] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c72209c0d8de44a0b6d16427eeae0aec" member_type: VOTER last_known_addr { host: "127.22.144.131" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:47:31.747995 12525 mvcc.cc:205] Tried to move back new op lower bound from 7317191277513601024 to 7317191277413519360. Current Snapshot: MvccSnapshot[applied={T|T < 7317191277513601024}]
I20260811 04:47:31.748708 12524 mvcc.cc:205] Tried to move back new op lower bound from 7317191277513601024 to 7317191277413519360. Current Snapshot: MvccSnapshot[applied={T|T < 7317191277513601024}]
I20260811 04:47:31.757808 11959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:31.758009 11959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 04:47:31.760831 11959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad1c1e8b1599473eb02ae7e37c2fcce9: Sending DeleteTablet for 3 replicas of tablet 40b4de0f163c4b58ba2bfa0b392e476f
I20260811 04:47:31.761799 12409 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b4de0f163c4b58ba2bfa0b392e476f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:31 UTC) from {username='slave'} at 127.0.0.1:55448
I20260811 04:47:31.761799 12102 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b4de0f163c4b58ba2bfa0b392e476f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:31 UTC) from {username='slave'} at 127.0.0.1:46714
I20260811 04:47:31.761935 12258 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b4de0f163c4b58ba2bfa0b392e476f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 04:47:31 UTC) from {username='slave'} at 127.0.0.1:46690
I20260811 04:47:31.762459 12537 tablet_replica.cc:330] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96: stopping tablet replica
I20260811 04:47:31.762573 12539 tablet_replica.cc:330] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: stopping tablet replica
I20260811 04:47:31.762830 12537 raft_consensus.cc:2229] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 1 LEADER]: Raft consensus shutting down.
I20260811 04:47:31.762890 12539 raft_consensus.cc:2229] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:31.762974 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12029
I20260811 04:47:31.763238 12539 raft_consensus.cc:2258] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:31.763276 12537 raft_consensus.cc:2258] T 40b4de0f163c4b58ba2bfa0b392e476f P 978ba619a8ed427881f56945940d0c96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:31.763808 12538 tablet_replica.cc:330] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: stopping tablet replica
I20260811 04:47:31.764102 12538 raft_consensus.cc:2229] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 04:47:31.764365 12538 raft_consensus.cc:2258] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 04:47:31.765729 12538 ts_tablet_manager.cc:1905] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:31.765729 12539 ts_tablet_manager.cc:1905] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 04:47:31.768576 12539 ts_tablet_manager.cc:1918] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:31.768576 12538 ts_tablet_manager.cc:1918] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 04:47:31.768687 12538 log.cc:1196] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-2/wal/wals/40b4de0f163c4b58ba2bfa0b392e476f
I20260811 04:47:31.768690 12539 log.cc:1196] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Deleting WAL directory at /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786423274535345-23106-0/minicluster-data/ts-1/wal/wals/40b4de0f163c4b58ba2bfa0b392e476f
I20260811 04:47:31.769023 12538 ts_tablet_manager.cc:1939] T 40b4de0f163c4b58ba2bfa0b392e476f P c72209c0d8de44a0b6d16427eeae0aec: Deleting consensus metadata
I20260811 04:47:31.769023 12539 ts_tablet_manager.cc:1939] T 40b4de0f163c4b58ba2bfa0b392e476f P 37378dada7cf4c97a05e7550b3c5975e: Deleting consensus metadata
I20260811 04:47:31.770407 11944 catalog_manager.cc:4841] TS c72209c0d8de44a0b6d16427eeae0aec (127.22.144.131:44195): tablet 40b4de0f163c4b58ba2bfa0b392e476f (table test-table [id=822a3d3e753443a3a3f98c32d0ec22ac]) successfully deleted
I20260811 04:47:31.770414 11945 catalog_manager.cc:4841] TS 37378dada7cf4c97a05e7550b3c5975e (127.22.144.130:35531): tablet 40b4de0f163c4b58ba2bfa0b392e476f (table test-table [id=822a3d3e753443a3a3f98c32d0ec22ac]) successfully deleted
W20260811 04:47:31.773324 11945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.144.129:37713 (error 108)
I20260811 04:47:31.774226 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12180
I20260811 04:47:31.774499 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:31.774146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.129:37713 (local address 127.0.0.1:46728)
0811 04:47:31.774292 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 04:47:31.774397 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.129:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 04:47:31.777185 11945 catalog_manager.cc:4584] TS 978ba619a8ed427881f56945940d0c96 (127.22.144.129:37713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40b4de0f163c4b58ba2bfa0b392e476f: Network error: Client connection negotiation failed: client connection to 127.22.144.129:37713: connect: Connection refused (error 111)
I20260811 04:47:31.782564 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12338
I20260811 04:47:31.790761 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 11928
2026-08-11T04:47:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1048 ms)
[----------] 6 tests from EncryptionPolicyTest (11329 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:31 dist-test-slave-ss4n krb5kdc[12543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:31 dist-test-slave-ss4n krb5kdc[12543](info): set up 2 sockets
Aug 11 04:47:31 dist-test-slave-ss4n krb5kdc[12543](info): commencing operation
krb5kdc: starting...
W20260811 04:47:32.917953 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:33 dist-test-slave-ss4n krb5kdc[12543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:33.061204 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44969
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:33.178210 12557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:33.178570 12557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:33.178669 12557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:33.182871 12557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:33.182991 12557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:33.183029 12557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:33.183073 12557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:33.183111 12557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:33.188771 12557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44969
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:33.190049 12557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:33.191164 12557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:33.200575 12566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:33.200569 12565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:33.200626 12568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:33.200932 12557 server_base.cc:1034] running on GCE node
I20260811 04:47:33.201682 12557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:33.202921 12557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:33.204097 12557 hybrid_clock.cc:648] HybridClock initialized: now 1786423653204082 us; error 28 us; skew 500 ppm
Aug 11 04:47:33 dist-test-slave-ss4n krb5kdc[12543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:33.207273 12557 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:33.208297 12557 webserver.cc:460] Webserver started at http://127.0.0.1:43039/ using document root <none> and password file <none>
I20260811 04:47:33.208896 12557 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:33.208994 12557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:33.209262 12557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:33.211591 12557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "0d4d66118914439a9b2a741088607d74"
format_stamp: "Formatted at 2026-08-11 04:47:33 on dist-test-slave-ss4n"
server_key: "034c303d50b6104d0305145e6250093d"
server_key_iv: "a430597c9a3a44735a53570e393844f1"
server_key_version: "encryptionkey@0"
I20260811 04:47:33.212229 12557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "0d4d66118914439a9b2a741088607d74"
format_stamp: "Formatted at 2026-08-11 04:47:33 on dist-test-slave-ss4n"
server_key: "034c303d50b6104d0305145e6250093d"
server_key_iv: "a430597c9a3a44735a53570e393844f1"
server_key_version: "encryptionkey@0"
I20260811 04:47:33.216514 12557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:33.219245 12575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:33.220556 12557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 04:47:33.220713 12557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "0d4d66118914439a9b2a741088607d74"
format_stamp: "Formatted at 2026-08-11 04:47:33 on dist-test-slave-ss4n"
server_key: "034c303d50b6104d0305145e6250093d"
server_key_iv: "a430597c9a3a44735a53570e393844f1"
server_key_version: "encryptionkey@0"
I20260811 04:47:33.220849 12557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:33.236546 12557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:33.239857 12557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:33.240095 12557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:33.248274 12557 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44969
I20260811 04:47:33.248279 12639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44969 every 8 connection(s)
I20260811 04:47:33.249405 12557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:33.252686 12640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:33.258752 12640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74: Bootstrap starting.
I20260811 04:47:33.259688 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12557
I20260811 04:47:33.259824 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:33.260192 23106 external_mini_cluster.cc:1428] Setting key 29661a177a9c3a67292f3e74487a2317
I20260811 04:47:33.261969 12640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:33.262770 12640 log.cc:824] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:33.265138 12640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74: No bootstrap required, opened a new log
Aug 11 04:47:33 dist-test-slave-ss4n krb5kdc[12543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 04:47:33.270066 12640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d4d66118914439a9b2a741088607d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44969 } }
I20260811 04:47:33.270344 12640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:33.270434 12640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4d66118914439a9b2a741088607d74, State: Initialized, Role: FOLLOWER
I20260811 04:47:33.270994 12640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d4d66118914439a9b2a741088607d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44969 } }
I20260811 04:47:33.271185 12640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:33.271296 12640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:33.271428 12640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:33.272392 12640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d4d66118914439a9b2a741088607d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44969 } }
I20260811 04:47:33.272766 12640 leader_election.cc:302] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d4d66118914439a9b2a741088607d74; no voters:
I20260811 04:47:33.273247 12640 leader_election.cc:288] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:33.273334 12646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:33.273645 12646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [term 1 LEADER]: Becoming Leader. State: Replica: 0d4d66118914439a9b2a741088607d74, State: Running, Role: LEADER
I20260811 04:47:33.274101 12646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d4d66118914439a9b2a741088607d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44969 } }
I20260811 04:47:33.274758 12640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:33.275503 12646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d4d66118914439a9b2a741088607d74. Latest consensus state: current_term: 1 leader_uuid: "0d4d66118914439a9b2a741088607d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d4d66118914439a9b2a741088607d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44969 } } }
I20260811 04:47:33.275699 12646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:33.276149 12647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d4d66118914439a9b2a741088607d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d4d66118914439a9b2a741088607d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44969 } } }
I20260811 04:47:33.276153 12652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:33.276396 12647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:33.276716 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:33.261451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48997 (local address 127.0.0.1:44969)
0811 04:47:33.261957 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:33.261973 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:33.262009 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:47:33.262753 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:33.262770 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:33.263219 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:33.263614 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:33.264514 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:33.265656 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:33.266844 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:33.267345 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:33.267491 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:33.270617 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:33.270623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:33.270641 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:33.270675 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:33.272701 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:33.273365 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:33.273372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:33.273380 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:33.273577 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:33.274061 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:33.274064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:33.274066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:33.274645 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:33.274942 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:33.275357 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 04:47:33.275615 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":146,"threads_started":1}
I20260811 04:47:33.279886 12652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:33.280613 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:33.286607 12652 catalog_manager.cc:1347] Generated new cluster ID: d392110c812d4f45a9f4af51f90f26f1
I20260811 04:47:33.286680 12652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:33.292923 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:33.283137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39155 (local address 127.0.0.1:44969)
0811 04:47:33.283267 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:33.283273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:33.283684 (+ 411us) server_negotiation.cc:407] Connection header received
0811 04:47:33.283838 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:33.283841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:33.283887 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:33.283980 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:33.284866 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:33.286201 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:33.287408 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:33.287742 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:33.287867 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:33.289384 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:33.289388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:33.289391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:33.289418 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:33.291048 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:33.291560 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:33.291563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:33.291565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:33.291653 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:33.291998 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:33.292001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:33.292003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:33.292190 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:33.292365 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:33.292653 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 04:47:33.292755 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260811 04:47:33.299269 12652 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:33.300408 12652 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:33.308127 12652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d4d66118914439a9b2a741088607d74: Generated new TSK 0
I20260811 04:47:33.308933 12652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:33.318756 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12557
W20260811 04:47:33.331193 12541 connection.cc:535] client connection to 127.0.0.1:44969 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44969): Connection reset by peer (error 104)
2026-08-11T04:47:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:33 dist-test-slave-ss4n krb5kdc[12675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:33 dist-test-slave-ss4n krb5kdc[12675](info): set up 2 sockets
Aug 11 04:47:33 dist-test-slave-ss4n krb5kdc[12675](info): commencing operation
krb5kdc: starting...
W20260811 04:47:34.514381 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:34 dist-test-slave-ss4n krb5kdc[12675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:34.656792 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:34.774564 12690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:34.774935 12690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:34.775022 12690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:34.779281 12690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:34.779397 12690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:34.779438 12690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:34.779496 12690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:34.779536 12690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:34.785754 12690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:34.786976 12690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:34.788093 12690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 04:47:34.797536 12690 server_base.cc:1034] running on GCE node
W20260811 04:47:34.797472 12701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:34.797456 12695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:34.797449 12696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:34.798259 12690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:34.799444 12690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:34.800683 12690 hybrid_clock.cc:648] HybridClock initialized: now 1786423654800654 us; error 49 us; skew 500 ppm
Aug 11 04:47:34 dist-test-slave-ss4n krb5kdc[12675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:34.809746 12690 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:34.810781 12690 webserver.cc:460] Webserver started at http://127.0.0.1:39685/ using document root <none> and password file <none>
I20260811 04:47:34.811406 12690 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:34.811510 12690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:34.811770 12690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:34.814216 12690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "766b43e71ffb4a66aa733d2716941334"
format_stamp: "Formatted at 2026-08-11 04:47:34 on dist-test-slave-ss4n"
server_key: "75197656fde71db93176190b322cfca4"
server_key_iv: "518ca4a3a3423b96ef1dde0470939a69"
server_key_version: "encryptionkey@0"
I20260811 04:47:34.814841 12690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "766b43e71ffb4a66aa733d2716941334"
format_stamp: "Formatted at 2026-08-11 04:47:34 on dist-test-slave-ss4n"
server_key: "75197656fde71db93176190b322cfca4"
server_key_iv: "518ca4a3a3423b96ef1dde0470939a69"
server_key_version: "encryptionkey@0"
I20260811 04:47:34.819034 12690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:34.821635 12708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:34.822995 12690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 04:47:34.823166 12690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "766b43e71ffb4a66aa733d2716941334"
format_stamp: "Formatted at 2026-08-11 04:47:34 on dist-test-slave-ss4n"
server_key: "75197656fde71db93176190b322cfca4"
server_key_iv: "518ca4a3a3423b96ef1dde0470939a69"
server_key_version: "encryptionkey@0"
I20260811 04:47:34.823292 12690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:34.838721 12690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:34.842353 12690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:34.842588 12690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:34.850601 12690 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40633
I20260811 04:47:34.850620 12768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40633 every 8 connection(s)
I20260811 04:47:34.851670 12690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:34.854815 12769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:34.855108 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12690
I20260811 04:47:34.855257 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:34.855587 23106 external_mini_cluster.cc:1428] Setting key 5f335c7cd7cd37931b5c33211806d68e
I20260811 04:47:34.862597 12769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334: Bootstrap starting.
I20260811 04:47:34.865429 12769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:34.866358 12769 log.cc:824] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:34.866442 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:34.857132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48093 (local address 127.0.0.1:40633)
0811 04:47:34.857639 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:34.857667 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:34.857714 (+ 47us) server_negotiation.cc:407] Connection header received
0811 04:47:34.858517 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:34.858545 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:34.859018 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:34.859426 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:34.859867 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:34.861434 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:34.862416 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:34.863097 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:34.863297 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:34.864298 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:34.864304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:34.864328 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:34.864826 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:34.864896 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:34.864931 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:34.865080 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:34.865284 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 04:47:34.865435 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":116,"threads_started":1}
I20260811 04:47:34.868553 12769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334: No bootstrap required, opened a new log
I20260811 04:47:34.869665 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:34.872848 12769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766b43e71ffb4a66aa733d2716941334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40633 } }
I20260811 04:47:34.873091 12769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:34.873150 12769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766b43e71ffb4a66aa733d2716941334, State: Initialized, Role: FOLLOWER
I20260811 04:47:34.873689 12769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766b43e71ffb4a66aa733d2716941334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40633 } }
I20260811 04:47:34.873864 12769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:34.873927 12769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:34.874029 12769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:34.875262 12769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766b43e71ffb4a66aa733d2716941334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40633 } }
I20260811 04:47:34.875686 12769 leader_election.cc:302] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 766b43e71ffb4a66aa733d2716941334; no voters:
I20260811 04:47:34.876112 12769 leader_election.cc:288] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:34.876241 12780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:34.876528 12780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [term 1 LEADER]: Becoming Leader. State: Replica: 766b43e71ffb4a66aa733d2716941334, State: Running, Role: LEADER
I20260811 04:47:34.876825 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:34.871849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40243 (local address 127.0.0.1:40633)
0811 04:47:34.871992 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:34.871998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:34.872187 (+ 189us) server_negotiation.cc:407] Connection header received
0811 04:47:34.872320 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:34.872322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:34.872359 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:34.872425 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:34.872743 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:34.873406 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:34.874596 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:34.874860 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:34.874971 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:34.876107 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:34.876111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:34.876114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:34.876129 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:34.876161 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:34.876176 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:34.876277 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:34.876632 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 04:47:34.876673 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:47:34.876947 12780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766b43e71ffb4a66aa733d2716941334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40633 } }
I20260811 04:47:34.877628 12769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:34.879007 12781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "766b43e71ffb4a66aa733d2716941334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766b43e71ffb4a66aa733d2716941334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40633 } } }
I20260811 04:47:34.879146 12781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:34.880014 12782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 766b43e71ffb4a66aa733d2716941334. Latest consensus state: current_term: 1 leader_uuid: "766b43e71ffb4a66aa733d2716941334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766b43e71ffb4a66aa733d2716941334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40633 } } }
I20260811 04:47:34.880160 12782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:34.880568 12790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:34.883469 12790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:34.889541 12790 catalog_manager.cc:1347] Generated new cluster ID: 348f8bf1a3554db0bf1b5f4acd215136
I20260811 04:47:34.889621 12790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:34.920857 12790 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:34.922034 12790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:34.930356 12790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 766b43e71ffb4a66aa733d2716941334: Generated new TSK 0
I20260811 04:47:34.931203 12790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:34.941342 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12690
W20260811 04:47:34.951500 12673 connection.cc:535] client connection to 127.0.0.1:40633 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40633): Connection reset by peer (error 104)
2026-08-11T04:47:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1620 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:35 dist-test-slave-ss4n krb5kdc[12804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:35 dist-test-slave-ss4n krb5kdc[12804](info): set up 2 sockets
Aug 11 04:47:35 dist-test-slave-ss4n krb5kdc[12804](info): commencing operation
krb5kdc: starting...
W20260811 04:47:36.073998 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:36 dist-test-slave-ss4n krb5kdc[12804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:36.216305 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38657
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:36.336449 12818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:36.336774 12818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:36.336822 12818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:36.341166 12818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:36.341302 12818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:36.341331 12818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:36.341351 12818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:36.341392 12818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:36.347314 12818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38657
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:36.348593 12818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:36.349831 12818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:36.359947 12824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:36.359947 12823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:36.360080 12827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:36.360342 12818 server_base.cc:1034] running on GCE node
I20260811 04:47:36.361246 12818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:36.362706 12818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:36.363936 12818 hybrid_clock.cc:648] HybridClock initialized: now 1786423656363906 us; error 46 us; skew 500 ppm
Aug 11 04:47:36 dist-test-slave-ss4n krb5kdc[12804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423656, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:36.367323 12818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:36.368374 12818 webserver.cc:460] Webserver started at http://127.0.0.1:34829/ using document root <none> and password file <none>
I20260811 04:47:36.368993 12818 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:36.369092 12818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:36.369345 12818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:36.371559 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "518eeee15e21424fb6d12eb5b89d3428"
format_stamp: "Formatted at 2026-08-11 04:47:36 on dist-test-slave-ss4n"
server_key: "80efff3b0342df3995bef4b94d56d3ce"
server_key_iv: "bfd600456433c5e00a6762903dafd361"
server_key_version: "encryptionkey@0"
I20260811 04:47:36.372179 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "518eeee15e21424fb6d12eb5b89d3428"
format_stamp: "Formatted at 2026-08-11 04:47:36 on dist-test-slave-ss4n"
server_key: "80efff3b0342df3995bef4b94d56d3ce"
server_key_iv: "bfd600456433c5e00a6762903dafd361"
server_key_version: "encryptionkey@0"
I20260811 04:47:36.376245 12818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:47:36.379097 12835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:36.380584 12818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:47:36.380764 12818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "518eeee15e21424fb6d12eb5b89d3428"
format_stamp: "Formatted at 2026-08-11 04:47:36 on dist-test-slave-ss4n"
server_key: "80efff3b0342df3995bef4b94d56d3ce"
server_key_iv: "bfd600456433c5e00a6762903dafd361"
server_key_version: "encryptionkey@0"
I20260811 04:47:36.380908 12818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:36.386166 12818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:36.389753 12818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:36.389912 12818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:36.397379 12818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38657
I20260811 04:47:36.397444 12899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38657 every 8 connection(s)
I20260811 04:47:36.398396 12818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:36.401280 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:36.404794 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12818
I20260811 04:47:36.404906 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:36.405217 23106 external_mini_cluster.cc:1428] Setting key aac5d5112968f513bf94de93677cf9e4
I20260811 04:47:36.407382 12900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428: Bootstrap starting.
I20260811 04:47:36.409854 12900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:36.410645 12900 log.cc:824] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:36.410821 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:36.406745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33821 (local address 127.0.0.1:38657)
0811 04:47:36.407192 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:36.407207 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:36.407242 (+ 35us) server_negotiation.cc:407] Connection header received
0811 04:47:36.408007 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:36.408027 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:36.408323 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:36.408684 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:36.409541 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:36.409552 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:36.409564 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:36.409575 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:36.409632 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:36.409652 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:36.409773 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:36.409878 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 04:47:36.410043 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":143,"threads_started":1}
I20260811 04:47:36.412920 12900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428: No bootstrap required, opened a new log
I20260811 04:47:36.413966 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:36.417174 12900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518eeee15e21424fb6d12eb5b89d3428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38657 } }
I20260811 04:47:36.417557 12900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:36.417637 12900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518eeee15e21424fb6d12eb5b89d3428, State: Initialized, Role: FOLLOWER
I20260811 04:47:36.418213 12900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518eeee15e21424fb6d12eb5b89d3428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38657 } }
I20260811 04:47:36.418370 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:36.416126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43141 (local address 127.0.0.1:38657)
0811 04:47:36.416269 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:36.416273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:36.416657 (+ 384us) server_negotiation.cc:407] Connection header received
0811 04:47:36.416829 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:36.416836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:36.416873 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:36.416963 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:36.417874 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:36.417877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:36.417880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:36.417897 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:36.417942 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:36.417954 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:36.418070 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:36.418151 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 04:47:36.418195 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 04:47:36.418443 12900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:36.418525 12900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:36.418648 12900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:36.419878 12900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518eeee15e21424fb6d12eb5b89d3428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38657 } }
I20260811 04:47:36.420210 12900 leader_election.cc:302] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 518eeee15e21424fb6d12eb5b89d3428; no voters:
I20260811 04:47:36.420485 12900 leader_election.cc:288] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:36.420743 12911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:36.421028 12911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [term 1 LEADER]: Becoming Leader. State: Replica: 518eeee15e21424fb6d12eb5b89d3428, State: Running, Role: LEADER
I20260811 04:47:36.421440 12911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518eeee15e21424fb6d12eb5b89d3428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38657 } }
I20260811 04:47:36.421914 12900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:36.423641 12914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 518eeee15e21424fb6d12eb5b89d3428. Latest consensus state: current_term: 1 leader_uuid: "518eeee15e21424fb6d12eb5b89d3428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518eeee15e21424fb6d12eb5b89d3428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38657 } } }
I20260811 04:47:36.423782 12914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:36.424103 12913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "518eeee15e21424fb6d12eb5b89d3428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "518eeee15e21424fb6d12eb5b89d3428" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38657 } } }
I20260811 04:47:36.424634 12913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:36.424221 12929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:36.427704 12929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:36.433413 12929 catalog_manager.cc:1347] Generated new cluster ID: 1f8e890c3ef64a428fa72c91e8ed4e07
I20260811 04:47:36.433524 12929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:36.464190 12929 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:36.464607 12929 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:36.472641 12929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 518eeee15e21424fb6d12eb5b89d3428: Generated new TSK 0
I20260811 04:47:36.473465 12929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:36.482133 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12818
2026-08-11T04:47:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:36 dist-test-slave-ss4n krb5kdc[12938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:36 dist-test-slave-ss4n krb5kdc[12938](info): set up 2 sockets
Aug 11 04:47:36 dist-test-slave-ss4n krb5kdc[12938](info): commencing operation
krb5kdc: starting...
W20260811 04:47:37.595432 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:37 dist-test-slave-ss4n krb5kdc[12938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:37.738153 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:37.856520 12952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:37.856825 12952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:37.856870 12952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:37.861152 12952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:37.861243 12952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:37.861263 12952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:37.861280 12952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:37.861296 12952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:37.867115 12952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:37.868311 12952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:37.869395 12952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:37.878342 12957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:37.878368 12952 server_base.cc:1034] running on GCE node
W20260811 04:47:37.878328 12959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:37.878633 12961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:37.879256 12952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:37.880445 12952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:37.881582 12952 hybrid_clock.cc:648] HybridClock initialized: now 1786423657881542 us; error 48 us; skew 500 ppm
Aug 11 04:47:37 dist-test-slave-ss4n krb5kdc[12938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:37.884423 12952 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:37.885255 12952 webserver.cc:460] Webserver started at http://127.0.0.1:46645/ using document root <none> and password file <none>
I20260811 04:47:37.885950 12952 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:37.886018 12952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:37.886188 12952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:37.888244 12952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee"
format_stamp: "Formatted at 2026-08-11 04:47:37 on dist-test-slave-ss4n"
server_key: "9f78ff916351bffd95cd7ecaf9efaed0"
server_key_iv: "dce2d7eb55e56a84408ac2230864f130"
server_key_version: "encryptionkey@0"
I20260811 04:47:37.888814 12952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee"
format_stamp: "Formatted at 2026-08-11 04:47:37 on dist-test-slave-ss4n"
server_key: "9f78ff916351bffd95cd7ecaf9efaed0"
server_key_iv: "dce2d7eb55e56a84408ac2230864f130"
server_key_version: "encryptionkey@0"
I20260811 04:47:37.892777 12952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 04:47:37.895309 12968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:37.896837 12952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:37.897012 12952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee"
format_stamp: "Formatted at 2026-08-11 04:47:37 on dist-test-slave-ss4n"
server_key: "9f78ff916351bffd95cd7ecaf9efaed0"
server_key_iv: "dce2d7eb55e56a84408ac2230864f130"
server_key_version: "encryptionkey@0"
I20260811 04:47:37.897152 12952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:37.924001 12952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:37.929250 12952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:37.929562 12952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:37.940145 12952 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38267
I20260811 04:47:37.940544 13025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38267 every 8 connection(s)
I20260811 04:47:37.941530 12952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:37.944873 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:37.946842 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 12952
I20260811 04:47:37.947013 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:37.947335 23106 external_mini_cluster.cc:1428] Setting key b552d5bb497b95d7bfe754e0d3c584fa
I20260811 04:47:37.952486 13026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee: Bootstrap starting.
I20260811 04:47:37.955524 13026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:37.956670 13026 log.cc:824] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:47:37 dist-test-slave-ss4n krb5kdc[12938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 04:47:37.959337 13026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee: No bootstrap required, opened a new log
I20260811 04:47:37.963743 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:37.948971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56859 (local address 127.0.0.1:38267)
0811 04:47:37.949526 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:37.949550 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:37.949600 (+ 50us) server_negotiation.cc:407] Connection header received
0811 04:47:37.950286 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:37.950304 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:37.950652 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:37.950944 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:37.952065 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:37.953171 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:37.954581 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:37.955096 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:37.955233 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:37.958738 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:37.958743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:37.958758 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:37.958786 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:37.960634 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:37.961202 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:37.961207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:37.961212 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:37.961326 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:37.961744 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:37.961747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:37.961748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:37.962152 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:37.962334 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:37.962647 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 04:47:37.962966 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":176,"threads_started":1}
I20260811 04:47:37.963971 13026 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260811 04:47:37.964243 13026 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:37.964329 13026 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e42d09cb3c49dc9f21f6e68ecc91ee, State: Initialized, Role: FOLLOWER
I20260811 04:47:37.964794 13026 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260811 04:47:37.964979 13026 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:37.965071 13026 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:37.965212 13026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:37.966454 13026 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260811 04:47:37.966840 13026 leader_election.cc:302] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9e42d09cb3c49dc9f21f6e68ecc91ee; no voters:
I20260811 04:47:37.967073 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:37.967253 13026 leader_election.cc:288] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:37.967373 13031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:37.967613 13031 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [term 1 LEADER]: Becoming Leader. State: Replica: e9e42d09cb3c49dc9f21f6e68ecc91ee, State: Running, Role: LEADER
I20260811 04:47:37.967979 13031 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260811 04:47:37.968770 13026 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:37.969753 13035 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9e42d09cb3c49dc9f21f6e68ecc91ee. Latest consensus state: current_term: 1 leader_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260811 04:47:37.969894 13035 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:37.970245 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e42d09cb3c49dc9f21f6e68ecc91ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260811 04:47:37.970325 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:37.971283 13051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:37.974716 13051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:37.979617 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:37.969545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57793 (local address 127.0.0.1:38267)
0811 04:47:37.969685 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:37.969692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:37.970320 (+ 628us) server_negotiation.cc:407] Connection header received
0811 04:47:37.970519 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:37.970523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:37.970585 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:37.970653 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:37.971772 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:37.972535 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:37.973675 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:37.974007 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:37.974107 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:37.976057 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:37.976060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:37.976063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:37.976090 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:37.977849 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:37.978309 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:37.978312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:37.978314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:37.978399 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:37.978700 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:37.978705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:37.978707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:37.978901 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:37.979045 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:37.979298 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 04:47:37.979416 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:47:37.982152 13051 catalog_manager.cc:1347] Generated new cluster ID: b34b354dd8fc412e835dd0c67b53bd12
I20260811 04:47:37.982244 13051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:38.000487 13051 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:38.001994 13051 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:38.013382 13051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9e42d09cb3c49dc9f21f6e68ecc91ee: Generated new TSK 0
I20260811 04:47:38.014400 13051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:38.022825 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 12952
2026-08-11T04:47:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:38 dist-test-slave-ss4n krb5kdc[13064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:38 dist-test-slave-ss4n krb5kdc[13064](info): set up 2 sockets
Aug 11 04:47:38 dist-test-slave-ss4n krb5kdc[13064](info): commencing operation
krb5kdc: starting...
W20260811 04:47:39.152817 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:39 dist-test-slave-ss4n krb5kdc[13064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:39.293120 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:39.414827 13077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:39.415171 13077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:39.415263 13077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:39.419581 13077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:39.419710 13077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:39.419745 13077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:39.419776 13077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:39.419811 13077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:39.425513 13077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:39.426718 13077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:39.427860 13077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:39.436993 13083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:39.436978 13084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:39.436977 13086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:39.437382 13077 server_base.cc:1034] running on GCE node
I20260811 04:47:39.438076 13077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:39.439301 13077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:39.440495 13077 hybrid_clock.cc:648] HybridClock initialized: now 1786423659440481 us; error 27 us; skew 500 ppm
Aug 11 04:47:39 dist-test-slave-ss4n krb5kdc[13064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:39.443662 13077 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:39.444645 13077 webserver.cc:460] Webserver started at http://127.0.0.1:32981/ using document root <none> and password file <none>
I20260811 04:47:39.445245 13077 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:39.445343 13077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:39.445650 13077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:39.447834 13077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "a9ac0c2f78d040cd866ad56b9318ed4b"
format_stamp: "Formatted at 2026-08-11 04:47:39 on dist-test-slave-ss4n"
server_key: "a53f2d1f95ae5a23f2f3dedd79bd460c"
server_key_iv: "a53d9b3cd6fe155347842dbd3fb014ba"
server_key_version: "encryptionkey@0"
I20260811 04:47:39.448436 13077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "a9ac0c2f78d040cd866ad56b9318ed4b"
format_stamp: "Formatted at 2026-08-11 04:47:39 on dist-test-slave-ss4n"
server_key: "a53f2d1f95ae5a23f2f3dedd79bd460c"
server_key_iv: "a53d9b3cd6fe155347842dbd3fb014ba"
server_key_version: "encryptionkey@0"
I20260811 04:47:39.452494 13077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 04:47:39.455538 13094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:39.457010 13077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:39.457129 13077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "a9ac0c2f78d040cd866ad56b9318ed4b"
format_stamp: "Formatted at 2026-08-11 04:47:39 on dist-test-slave-ss4n"
server_key: "a53f2d1f95ae5a23f2f3dedd79bd460c"
server_key_iv: "a53d9b3cd6fe155347842dbd3fb014ba"
server_key_version: "encryptionkey@0"
I20260811 04:47:39.457233 13077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:39.472790 13077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:39.476218 13077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:39.476390 13077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:39.483798 13077 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36487
I20260811 04:47:39.483825 13152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36487 every 8 connection(s)
I20260811 04:47:39.484764 13077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:39.487533 13153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:39.491257 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13077
I20260811 04:47:39.491413 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:39.491743 23106 external_mini_cluster.cc:1428] Setting key 8f150735bf847009d8d9f4f753976c26
I20260811 04:47:39.493854 13153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b: Bootstrap starting.
I20260811 04:47:39.496079 13153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:39.496855 13153 log.cc:824] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:39.498811 13153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b: No bootstrap required, opened a new log
I20260811 04:47:39.501289 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:39.493352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42121 (local address 127.0.0.1:36487)
0811 04:47:39.493836 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:39.493851 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:39.493888 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:47:39.494701 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:39.494717 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:39.495066 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:39.495411 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:39.495872 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:39.496955 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:39.497963 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:39.498443 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:39.498657 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:39.499583 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:39.499588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:39.499603 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:39.500021 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:39.500100 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:39.500124 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:39.500232 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:39.500368 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 04:47:39.500585 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":163,"threads_started":1}
I20260811 04:47:39.502588 13153 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 04:47:39.502838 13153 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:39.502920 13153 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ac0c2f78d040cd866ad56b9318ed4b, State: Initialized, Role: FOLLOWER
I20260811 04:47:39.503433 13153 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 04:47:39.503602 13153 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:39.503672 13153 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:39.503778 13153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:39.504225 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:39.504984 13153 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 04:47:39.505380 13153 leader_election.cc:302] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9ac0c2f78d040cd866ad56b9318ed4b; no voters:
I20260811 04:47:39.505775 13153 leader_election.cc:288] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:39.505949 13160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:39.506233 13160 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [term 1 LEADER]: Becoming Leader. State: Replica: a9ac0c2f78d040cd866ad56b9318ed4b, State: Running, Role: LEADER
I20260811 04:47:39.506629 13160 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 04:47:39.507184 13153 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:39.509362 13164 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } } }
I20260811 04:47:39.509527 13164 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:39.509603 13163 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9ac0c2f78d040cd866ad56b9318ed4b. Latest consensus state: current_term: 1 leader_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ac0c2f78d040cd866ad56b9318ed4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } } }
I20260811 04:47:39.509692 13163 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:39.511730 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:39.506878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54991 (local address 127.0.0.1:36487)
0811 04:47:39.506979 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:39.506983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:39.507098 (+ 115us) server_negotiation.cc:407] Connection header received
0811 04:47:39.507269 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:39.507274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:39.507327 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:39.507426 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:39.507842 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:39.508653 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:39.509836 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:39.510214 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:39.510325 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:39.511265 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:39.511268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:39.511269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:39.511282 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:39.511308 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:39.511318 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:39.511381 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:39.511521 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 04:47:39.511572 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 04:47:39.513243 13182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:47:39.513336 13182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:47:39.513446 13183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:39.514243 13183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:39.520650 13183 catalog_manager.cc:1347] Generated new cluster ID: 1225f65aa51b48bf811b22a88bdd28cc
I20260811 04:47:39.520756 13183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:39.537926 13183 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:39.539062 13183 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:39.554342 13183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9ac0c2f78d040cd866ad56b9318ed4b: Generated new TSK 0
I20260811 04:47:39.555193 13183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:39.567447 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13077
2026-08-11T04:47:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1542 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:39 dist-test-slave-ss4n krb5kdc[13190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:39 dist-test-slave-ss4n krb5kdc[13190](info): set up 2 sockets
Aug 11 04:47:39 dist-test-slave-ss4n krb5kdc[13190](info): commencing operation
krb5kdc: starting...
W20260811 04:47:40.697713 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:40 dist-test-slave-ss4n krb5kdc[13190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:40.841436 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:40.960453 13206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:40.960762 13206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:40.960840 13206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:40.965395 13206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:40.965528 13206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:40.965564 13206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:40.965631 13206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:40.965667 13206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:40.971668 13206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:40.972833 13206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:40.974026 13206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:40.983579 13212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:40.983614 13214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:40.983572 13211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:40.983978 13206 server_base.cc:1034] running on GCE node
I20260811 04:47:40.984670 13206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:40.985984 13206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:40.987200 13206 hybrid_clock.cc:648] HybridClock initialized: now 1786423660987187 us; error 29 us; skew 500 ppm
Aug 11 04:47:40 dist-test-slave-ss4n krb5kdc[13190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:40.994257 13206 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:40.995344 13206 webserver.cc:460] Webserver started at http://127.0.0.1:46811/ using document root <none> and password file <none>
I20260811 04:47:40.995939 13206 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:40.996006 13206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:40.996205 13206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:40.998540 13206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "ec6cc172f9f64b92848e44612079dbf3"
format_stamp: "Formatted at 2026-08-11 04:47:40 on dist-test-slave-ss4n"
server_key: "322f940e0dbdca35e37afd4c8a858de2"
server_key_iv: "5fb7e073449dec5d514eaf6687b26c46"
server_key_version: "encryptionkey@0"
I20260811 04:47:40.999198 13206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "ec6cc172f9f64b92848e44612079dbf3"
format_stamp: "Formatted at 2026-08-11 04:47:40 on dist-test-slave-ss4n"
server_key: "322f940e0dbdca35e37afd4c8a858de2"
server_key_iv: "5fb7e073449dec5d514eaf6687b26c46"
server_key_version: "encryptionkey@0"
I20260811 04:47:41.003583 13206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:41.006229 13222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:41.007715 13206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:41.007840 13206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "ec6cc172f9f64b92848e44612079dbf3"
format_stamp: "Formatted at 2026-08-11 04:47:40 on dist-test-slave-ss4n"
server_key: "322f940e0dbdca35e37afd4c8a858de2"
server_key_iv: "5fb7e073449dec5d514eaf6687b26c46"
server_key_version: "encryptionkey@0"
I20260811 04:47:41.007961 13206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:41.012974 13206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:41.016201 13206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:41.016363 13206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:41.024290 13206 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33529
I20260811 04:47:41.024344 13282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33529 every 8 connection(s)
I20260811 04:47:41.025316 13206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:41.028425 13283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:41.029361 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13206
I20260811 04:47:41.029549 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:41.029873 23106 external_mini_cluster.cc:1428] Setting key 1805be242797e01fc950d766a0afa7c8
I20260811 04:47:41.035871 13283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3: Bootstrap starting.
I20260811 04:47:41.035974 13287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:41.031280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48541 (local address 127.0.0.1:33529)
0811 04:47:41.031745 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:41.031765 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:41.031885 (+ 120us) server_negotiation.cc:407] Connection header received
0811 04:47:41.032813 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:41.032837 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:41.033214 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:41.033652 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:41.034460 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:41.034470 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:41.034489 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:41.034502 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:41.034577 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:41.034606 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:41.034764 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:41.034843 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 04:47:41.035012 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":103,"threads_started":1}
I20260811 04:47:41.038678 13283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:41.039275 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:41.039636 13283 log.cc:824] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:41.042042 13283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3: No bootstrap required, opened a new log
I20260811 04:47:41.044318 13287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:41.042601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47887 (local address 127.0.0.1:33529)
0811 04:47:41.042734 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:41.042739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:41.042959 (+ 220us) server_negotiation.cc:407] Connection header received
0811 04:47:41.043073 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:41.043076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:41.043113 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:41.043197 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:41.043928 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:41.043931 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:41.043934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:41.043948 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:41.043980 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:41.043993 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:41.044094 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:41.044147 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 04:47:41.044189 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 04:47:41.046726 13283 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6cc172f9f64b92848e44612079dbf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33529 } }
I20260811 04:47:41.046976 13283 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:41.047055 13283 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec6cc172f9f64b92848e44612079dbf3, State: Initialized, Role: FOLLOWER
I20260811 04:47:41.047577 13283 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6cc172f9f64b92848e44612079dbf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33529 } }
I20260811 04:47:41.047741 13283 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:41.047818 13283 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:41.047930 13283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:41.049057 13283 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6cc172f9f64b92848e44612079dbf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33529 } }
I20260811 04:47:41.049424 13283 leader_election.cc:302] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec6cc172f9f64b92848e44612079dbf3; no voters:
I20260811 04:47:41.049824 13283 leader_election.cc:288] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:41.049932 13295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:41.050251 13295 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [term 1 LEADER]: Becoming Leader. State: Replica: ec6cc172f9f64b92848e44612079dbf3, State: Running, Role: LEADER
I20260811 04:47:41.050669 13295 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6cc172f9f64b92848e44612079dbf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33529 } }
I20260811 04:47:41.051235 13283 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:41.052492 13298 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec6cc172f9f64b92848e44612079dbf3. Latest consensus state: current_term: 1 leader_uuid: "ec6cc172f9f64b92848e44612079dbf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6cc172f9f64b92848e44612079dbf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33529 } } }
I20260811 04:47:41.052641 13298 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:41.052986 13296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec6cc172f9f64b92848e44612079dbf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec6cc172f9f64b92848e44612079dbf3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33529 } } }
I20260811 04:47:41.053078 13305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:41.053110 13296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:41.056347 13305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:41.062562 13305 catalog_manager.cc:1347] Generated new cluster ID: 951721b87d3d4ddb821636985c29c49c
I20260811 04:47:41.062649 13305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:41.076728 13305 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:41.077098 13305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:41.082535 13305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec6cc172f9f64b92848e44612079dbf3: Generated new TSK 0
I20260811 04:47:41.083391 13305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:41.088785 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13206
2026-08-11T04:47:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:41 dist-test-slave-ss4n krb5kdc[13321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:41 dist-test-slave-ss4n krb5kdc[13321](info): set up 2 sockets
Aug 11 04:47:41 dist-test-slave-ss4n krb5kdc[13321](info): commencing operation
krb5kdc: starting...
W20260811 04:47:42.223067 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:42 dist-test-slave-ss4n krb5kdc[13321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:42.365522 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:42.483435 13335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:42.483723 13335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:42.483767 13335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:42.487680 13335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:42.487736 13335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:42.487756 13335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:42.487772 13335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:42.487787 13335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:42.493031 13335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:42.494088 13335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:42.495093 13335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:42.504447 13344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:42.504451 13346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:42.504364 13342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:42.504840 13335 server_base.cc:1034] running on GCE node
I20260811 04:47:42.505420 13335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:42.506629 13335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:42.507838 13335 hybrid_clock.cc:648] HybridClock initialized: now 1786423662507819 us; error 42 us; skew 500 ppm
Aug 11 04:47:42 dist-test-slave-ss4n krb5kdc[13321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:42.510972 13335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:42.511978 13335 webserver.cc:460] Webserver started at http://127.0.0.1:33099/ using document root <none> and password file <none>
I20260811 04:47:42.512677 13335 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:42.512775 13335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:42.513020 13335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:42.515201 13335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "e58229b277184b82abed1e684c1edf0c"
format_stamp: "Formatted at 2026-08-11 04:47:42 on dist-test-slave-ss4n"
server_key: "4b4434e20fa5bd5ec34caea9ee855a72"
server_key_iv: "7b05759fbc3b5acbf230df078a5aa8f9"
server_key_version: "encryptionkey@0"
I20260811 04:47:42.515810 13335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "e58229b277184b82abed1e684c1edf0c"
format_stamp: "Formatted at 2026-08-11 04:47:42 on dist-test-slave-ss4n"
server_key: "4b4434e20fa5bd5ec34caea9ee855a72"
server_key_iv: "7b05759fbc3b5acbf230df078a5aa8f9"
server_key_version: "encryptionkey@0"
I20260811 04:47:42.519871 13335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:42.522536 13354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:42.523928 13335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:42.524274 13335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "e58229b277184b82abed1e684c1edf0c"
format_stamp: "Formatted at 2026-08-11 04:47:42 on dist-test-slave-ss4n"
server_key: "4b4434e20fa5bd5ec34caea9ee855a72"
server_key_iv: "7b05759fbc3b5acbf230df078a5aa8f9"
server_key_version: "encryptionkey@0"
I20260811 04:47:42.524462 13335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:42.550829 13335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:42.555275 13335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:42.555526 13335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:42.565052 13335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41537
I20260811 04:47:42.565083 13411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41537 every 8 connection(s)
I20260811 04:47:42.566366 13335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:42.569742 13412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:42.573823 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13335
I20260811 04:47:42.573988 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:42.574306 23106 external_mini_cluster.cc:1428] Setting key 616e1ec8258f9774e9668483c4af7058
I20260811 04:47:42.576537 13412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c: Bootstrap starting.
I20260811 04:47:42.579545 13412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:42.580416 13412 log.cc:824] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:42.583077 13412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c: No bootstrap required, opened a new log
Aug 11 04:47:42 dist-test-slave-ss4n krb5kdc[13321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 04:47:42.587134 13412 raft_consensus.cc:348] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58229b277184b82abed1e684c1edf0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41537 } }
I20260811 04:47:42.587469 13412 raft_consensus.cc:374] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:42.587567 13412 raft_consensus.cc:729] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58229b277184b82abed1e684c1edf0c, State: Initialized, Role: FOLLOWER
I20260811 04:47:42.588101 13412 consensus_queue.cc:260] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58229b277184b82abed1e684c1edf0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41537 } }
I20260811 04:47:42.588253 13412 raft_consensus.cc:388] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:42.588373 13412 raft_consensus.cc:482] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:42.588529 13412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:42.589644 13412 raft_consensus.cc:504] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58229b277184b82abed1e684c1edf0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41537 } }
I20260811 04:47:42.590090 13412 leader_election.cc:302] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e58229b277184b82abed1e684c1edf0c; no voters:
I20260811 04:47:42.590523 13412 leader_election.cc:288] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:42.590742 13419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:42.590988 13419 raft_consensus.cc:686] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [term 1 LEADER]: Becoming Leader. State: Replica: e58229b277184b82abed1e684c1edf0c, State: Running, Role: LEADER
I20260811 04:47:42.591398 13419 consensus_queue.cc:237] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58229b277184b82abed1e684c1edf0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41537 } }
I20260811 04:47:42.592222 13412 sys_catalog.cc:564] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:42.592864 13419 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e58229b277184b82abed1e684c1edf0c. Latest consensus state: current_term: 1 leader_uuid: "e58229b277184b82abed1e684c1edf0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58229b277184b82abed1e684c1edf0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41537 } } }
I20260811 04:47:42.592989 13419 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:42.593366 13420 sys_catalog.cc:455] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e58229b277184b82abed1e684c1edf0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e58229b277184b82abed1e684c1edf0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41537 } } }
I20260811 04:47:42.593528 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:42.575947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:49189 (local address 127.0.0.1:41537)
0811 04:47:42.576436 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:42.576453 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:42.576492 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:47:42.577276 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:42.577296 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:42.577758 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:42.578081 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:42.579182 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:42.580403 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:42.581648 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:42.582200 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:42.582346 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:42.585629 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:42.585645 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:42.585662 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:42.585695 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:42.588861 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:42.589516 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:42.589523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:42.589529 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:42.589678 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:42.590156 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:42.590159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:42.590161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:42.590817 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:42.591832 (+ 1015us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:42.592029 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 04:47:42.592299 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":175,"threads_started":1}
I20260811 04:47:42.593531 13420 sys_catalog.cc:458] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:42.595228 13426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:42.597532 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:42.599462 13426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:42.606914 13426 catalog_manager.cc:1347] Generated new cluster ID: a6b56cc32b624b84ace4234a482987c7
I20260811 04:47:42.607017 13426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:42.615504 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:42.599725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:60167 (local address 127.0.0.1:41537)
0811 04:47:42.599870 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:42.599875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:42.604739 (+ 4864us) server_negotiation.cc:407] Connection header received
0811 04:47:42.604984 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:42.604989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:42.605063 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:42.605176 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:42.606362 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:42.607282 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:42.608584 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:42.609018 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:42.609144 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:42.611148 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:42.611152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:42.611156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:42.611189 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:42.613189 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:42.613877 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:42.613881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:42.613883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:42.613987 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:42.614390 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:42.614394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:42.614396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:42.614677 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:42.614840 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:42.615190 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 04:47:42.615309 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:47:42.624759 13426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:42.626226 13426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:42.642309 13426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e58229b277184b82abed1e684c1edf0c: Generated new TSK 0
I20260811 04:47:42.643113 13426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:42.649735 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13335
W20260811 04:47:42.659524 13319 connection.cc:535] client connection to 127.0.0.1:41537 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41537): Connection reset by peer (error 104)
2026-08-11T04:47:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:42 dist-test-slave-ss4n krb5kdc[13448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:42 dist-test-slave-ss4n krb5kdc[13448](info): set up 2 sockets
Aug 11 04:47:42 dist-test-slave-ss4n krb5kdc[13448](info): commencing operation
krb5kdc: starting...
W20260811 04:47:43.783671 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:43 dist-test-slave-ss4n krb5kdc[13448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:43.929350 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:44.050190 13462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:44.050540 13462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:44.050623 13462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:44.054785 13462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:44.054898 13462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:44.054937 13462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:44.054978 13462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:44.055015 13462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:44.060741 13462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:44.062014 13462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:44.063226 13462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:44.072535 13468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:44.072553 13469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:44.072654 13471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:44.072769 13462 server_base.cc:1034] running on GCE node
I20260811 04:47:44.073580 13462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:44.074900 13462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:44.076108 13462 hybrid_clock.cc:648] HybridClock initialized: now 1786423664076080 us; error 49 us; skew 500 ppm
Aug 11 04:47:44 dist-test-slave-ss4n krb5kdc[13448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423664, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:44.079407 13462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:44.080365 13462 webserver.cc:460] Webserver started at http://127.0.0.1:34411/ using document root <none> and password file <none>
I20260811 04:47:44.081032 13462 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:44.081136 13462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:44.081379 13462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:44.083563 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "978b76e086924763af6ec626a64bb8d6"
format_stamp: "Formatted at 2026-08-11 04:47:44 on dist-test-slave-ss4n"
server_key: "9c61b10115e080ddf57ccded101f0a1e"
server_key_iv: "e8212fd4ec9b365f4f37a01494b6ed7d"
server_key_version: "encryptionkey@0"
I20260811 04:47:44.084143 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "978b76e086924763af6ec626a64bb8d6"
format_stamp: "Formatted at 2026-08-11 04:47:44 on dist-test-slave-ss4n"
server_key: "9c61b10115e080ddf57ccded101f0a1e"
server_key_iv: "e8212fd4ec9b365f4f37a01494b6ed7d"
server_key_version: "encryptionkey@0"
I20260811 04:47:44.088155 13462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:47:44.090907 13480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:44.092343 13462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:47:44.092497 13462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "978b76e086924763af6ec626a64bb8d6"
format_stamp: "Formatted at 2026-08-11 04:47:44 on dist-test-slave-ss4n"
server_key: "9c61b10115e080ddf57ccded101f0a1e"
server_key_iv: "e8212fd4ec9b365f4f37a01494b6ed7d"
server_key_version: "encryptionkey@0"
I20260811 04:47:44.092628 13462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:44.110700 13462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:44.114336 13462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:44.114573 13462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:44.122606 13462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46627
I20260811 04:47:44.122637 13541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46627 every 8 connection(s)
I20260811 04:47:44.123667 13462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:44.126881 13542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:44.128152 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13462
I20260811 04:47:44.128264 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:44.128579 23106 external_mini_cluster.cc:1428] Setting key b64b9b2b3fcaaaf7df56e7c73a352034
I20260811 04:47:44.135073 13542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6: Bootstrap starting.
I20260811 04:47:44.138162 13542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:44.139259 13542 log.cc:824] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:44.140376 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:44.130272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:50365 (local address 127.0.0.1:46627)
0811 04:47:44.130748 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:44.130770 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:44.130820 (+ 50us) server_negotiation.cc:407] Connection header received
0811 04:47:44.131750 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:44.131779 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:44.132265 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:44.132691 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:44.133149 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:44.134816 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:44.135887 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:44.136692 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:44.136887 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:44.138037 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:44.138047 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:44.138073 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:44.138665 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:44.138743 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:44.138777 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:44.138931 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:44.139187 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 04:47:44.139347 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260811 04:47:44.141775 13542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6: No bootstrap required, opened a new log
I20260811 04:47:44.143874 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:44.146600 13542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978b76e086924763af6ec626a64bb8d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46627 } }
I20260811 04:47:44.146895 13542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:44.147003 13542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978b76e086924763af6ec626a64bb8d6, State: Initialized, Role: FOLLOWER
I20260811 04:47:44.147537 13542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978b76e086924763af6ec626a64bb8d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46627 } }
I20260811 04:47:44.147715 13542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:44.147799 13542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:44.147944 13542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:44.149107 13542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978b76e086924763af6ec626a64bb8d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46627 } }
I20260811 04:47:44.149590 13542 leader_election.cc:302] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 978b76e086924763af6ec626a64bb8d6; no voters:
I20260811 04:47:44.150029 13542 leader_election.cc:288] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:44.150130 13552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:44.150377 13552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [term 1 LEADER]: Becoming Leader. State: Replica: 978b76e086924763af6ec626a64bb8d6, State: Running, Role: LEADER
I20260811 04:47:44.150545 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:44.146102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:38787 (local address 127.0.0.1:46627)
0811 04:47:44.146243 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:44.146250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:44.146456 (+ 206us) server_negotiation.cc:407] Connection header received
0811 04:47:44.146620 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:44.146623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:44.146671 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:44.146764 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:44.147084 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:44.147624 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:44.148472 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:44.148747 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:44.148862 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:44.149905 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:44.149911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:44.149916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:44.149937 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:44.149975 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:44.149990 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:44.150110 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:44.150347 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 04:47:44.150395 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:47:44.150738 13552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978b76e086924763af6ec626a64bb8d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46627 } }
I20260811 04:47:44.151553 13542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:44.152081 13552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 978b76e086924763af6ec626a64bb8d6. Latest consensus state: current_term: 1 leader_uuid: "978b76e086924763af6ec626a64bb8d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978b76e086924763af6ec626a64bb8d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46627 } } }
I20260811 04:47:44.152205 13552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:44.152587 13558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:44.152810 13553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "978b76e086924763af6ec626a64bb8d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978b76e086924763af6ec626a64bb8d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46627 } } }
I20260811 04:47:44.153012 13553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:44.155918 13558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:44.163316 13558 catalog_manager.cc:1347] Generated new cluster ID: 81c82b895751423ea8edd5c9bd6e65ba
I20260811 04:47:44.163414 13558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:44.188078 13558 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:44.189369 13558 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:44.198051 13558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 978b76e086924763af6ec626a64bb8d6: Generated new TSK 0
I20260811 04:47:44.198916 13558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:44.207139 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13462
W20260811 04:47:44.217710 13446 connection.cc:535] client connection to 127.0.0.1:46627 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46627): Connection reset by peer (error 104)
2026-08-11T04:47:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:44 dist-test-slave-ss4n krb5kdc[13575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:44 dist-test-slave-ss4n krb5kdc[13575](info): set up 2 sockets
Aug 11 04:47:44 dist-test-slave-ss4n krb5kdc[13575](info): commencing operation
krb5kdc: starting...
W20260811 04:47:45.375057 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:45 dist-test-slave-ss4n krb5kdc[13575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:45.517212 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:45.637835 13592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:45.638218 13592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:45.638312 13592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:45.642549 13592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:45.642661 13592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:45.642696 13592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:45.642733 13592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:45.642769 13592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:45.648438 13592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:45.649659 13592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:45.650727 13592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:45.659807 13598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:45.659812 13597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:45.660025 13600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:45.660681 13592 server_base.cc:1034] running on GCE node
I20260811 04:47:45.661291 13592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:45.662515 13592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:45.663738 13592 hybrid_clock.cc:648] HybridClock initialized: now 1786423665663725 us; error 28 us; skew 500 ppm
Aug 11 04:47:45 dist-test-slave-ss4n krb5kdc[13575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:45.666795 13592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:45.667847 13592 webserver.cc:460] Webserver started at http://127.0.0.1:45731/ using document root <none> and password file <none>
I20260811 04:47:45.668454 13592 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:45.668555 13592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:45.668795 13592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:45.671052 13592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "4c8de739390f4922ae7ff7d91e9b703b"
format_stamp: "Formatted at 2026-08-11 04:47:45 on dist-test-slave-ss4n"
server_key: "0a0f4266bd6e5a546f862a63a9107d96"
server_key_iv: "c46a2eecebf7aa2fd095951666dd5c5f"
server_key_version: "encryptionkey@0"
I20260811 04:47:45.671670 13592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "4c8de739390f4922ae7ff7d91e9b703b"
format_stamp: "Formatted at 2026-08-11 04:47:45 on dist-test-slave-ss4n"
server_key: "0a0f4266bd6e5a546f862a63a9107d96"
server_key_iv: "c46a2eecebf7aa2fd095951666dd5c5f"
server_key_version: "encryptionkey@0"
I20260811 04:47:45.675863 13592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:45.678557 13607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:45.679913 13592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:45.680081 13592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "4c8de739390f4922ae7ff7d91e9b703b"
format_stamp: "Formatted at 2026-08-11 04:47:45 on dist-test-slave-ss4n"
server_key: "0a0f4266bd6e5a546f862a63a9107d96"
server_key_iv: "c46a2eecebf7aa2fd095951666dd5c5f"
server_key_version: "encryptionkey@0"
I20260811 04:47:45.680217 13592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:45.685225 13592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:45.688666 13592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:45.688908 13592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:45.696820 13592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43199
I20260811 04:47:45.696851 13669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43199 every 8 connection(s)
I20260811 04:47:45.697875 13592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:45.700851 13670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:45.705255 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13592
I20260811 04:47:45.705421 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:45.705801 23106 external_mini_cluster.cc:1428] Setting key 2025684c9744707e45ac0049833a57bc
I20260811 04:47:45.707527 13670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b: Bootstrap starting.
I20260811 04:47:45.710114 13670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:45.711248 13670 log.cc:824] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:45.711585 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:45.707391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:43419 (local address 127.0.0.1:43199)
0811 04:47:45.707918 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:45.707934 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:45.707972 (+ 38us) server_negotiation.cc:407] Connection header received
0811 04:47:45.708649 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:45.708676 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:45.708977 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:45.709343 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:45.710240 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:45.710249 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:45.710266 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:45.710279 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:45.710350 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:45.710373 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:45.710497 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:45.710609 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 04:47:45.710798 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":163,"threads_started":1}
I20260811 04:47:45.713593 13670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b: No bootstrap required, opened a new log
I20260811 04:47:45.714691 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:45.717975 13670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8de739390f4922ae7ff7d91e9b703b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260811 04:47:45.718329 13670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:45.718415 13670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c8de739390f4922ae7ff7d91e9b703b, State: Initialized, Role: FOLLOWER
I20260811 04:47:45.718952 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:45.716955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:34321 (local address 127.0.0.1:43199)
0811 04:47:45.717104 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:45.717108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:45.717342 (+ 234us) server_negotiation.cc:407] Connection header received
0811 04:47:45.717472 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:45.717474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:45.717556 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:45.717640 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:45.718502 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:45.718506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:45.718509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:45.718522 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:45.718552 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:45.718572 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:45.718645 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:45.718767 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 04:47:45.718811 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:47:45.718979 13670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8de739390f4922ae7ff7d91e9b703b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260811 04:47:45.719163 13670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:45.719244 13670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:45.719357 13670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:45.720504 13670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8de739390f4922ae7ff7d91e9b703b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260811 04:47:45.720827 13670 leader_election.cc:302] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c8de739390f4922ae7ff7d91e9b703b; no voters:
I20260811 04:47:45.721099 13670 leader_election.cc:288] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:45.721228 13680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:45.721536 13680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [term 1 LEADER]: Becoming Leader. State: Replica: 4c8de739390f4922ae7ff7d91e9b703b, State: Running, Role: LEADER
I20260811 04:47:45.721902 13680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8de739390f4922ae7ff7d91e9b703b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260811 04:47:45.722537 13670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:45.723129 13680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c8de739390f4922ae7ff7d91e9b703b. Latest consensus state: current_term: 1 leader_uuid: "4c8de739390f4922ae7ff7d91e9b703b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8de739390f4922ae7ff7d91e9b703b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } } }
I20260811 04:47:45.723253 13680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:45.724201 13690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:45.725087 13681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c8de739390f4922ae7ff7d91e9b703b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8de739390f4922ae7ff7d91e9b703b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } } }
I20260811 04:47:45.725191 13681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:45.728295 13690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:45.734825 13690 catalog_manager.cc:1347] Generated new cluster ID: 633fb6917e4b41e5980af4636e73da00
I20260811 04:47:45.734910 13690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:45.767302 13690 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:45.767750 13690 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:45.774174 13690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c8de739390f4922ae7ff7d91e9b703b: Generated new TSK 0
I20260811 04:47:45.775010 13690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:45.778674 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13592
2026-08-11T04:47:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:45 dist-test-slave-ss4n krb5kdc[13708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:45 dist-test-slave-ss4n krb5kdc[13708](info): set up 2 sockets
Aug 11 04:47:45 dist-test-slave-ss4n krb5kdc[13708](info): commencing operation
krb5kdc: starting...
W20260811 04:47:46.993705 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:47 dist-test-slave-ss4n krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:47.132817 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:47.250319 13721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:47.250615 13721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:47.250660 13721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:47.254786 13721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:47.254855 13721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:47.254876 13721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:47.254894 13721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:47.254908 13721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:47.260486 13721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37677
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:47.261649 13721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:47.262737 13721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:47.272588 13729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:47.272768 13721 server_base.cc:1034] running on GCE node
W20260811 04:47:47.272629 13727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:47.272647 13726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:47.273589 13721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:47.274893 13721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:47.276111 13721 hybrid_clock.cc:648] HybridClock initialized: now 1786423667276093 us; error 32 us; skew 500 ppm
Aug 11 04:47:47 dist-test-slave-ss4n krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:47.283905 13721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:47.284948 13721 webserver.cc:460] Webserver started at http://127.0.0.1:45687/ using document root <none> and password file <none>
I20260811 04:47:47.285594 13721 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:47.285663 13721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:47.285840 13721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:47.287942 13721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "2fbb55396ed949429eb42a6e8b01f27e"
format_stamp: "Formatted at 2026-08-11 04:47:47 on dist-test-slave-ss4n"
server_key: "c1cd94ee83090efe7fd0b6ba85dfb7f7"
server_key_iv: "1e8336a97761b39d4affe63802ad3556"
server_key_version: "encryptionkey@0"
I20260811 04:47:47.288491 13721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "2fbb55396ed949429eb42a6e8b01f27e"
format_stamp: "Formatted at 2026-08-11 04:47:47 on dist-test-slave-ss4n"
server_key: "c1cd94ee83090efe7fd0b6ba85dfb7f7"
server_key_iv: "1e8336a97761b39d4affe63802ad3556"
server_key_version: "encryptionkey@0"
I20260811 04:47:47.292368 13721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 04:47:47.295120 13737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:47.296460 13721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:47:47.296617 13721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "2fbb55396ed949429eb42a6e8b01f27e"
format_stamp: "Formatted at 2026-08-11 04:47:47 on dist-test-slave-ss4n"
server_key: "c1cd94ee83090efe7fd0b6ba85dfb7f7"
server_key_iv: "1e8336a97761b39d4affe63802ad3556"
server_key_version: "encryptionkey@0"
I20260811 04:47:47.296723 13721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:47.325471 13721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:47.329411 13721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:47.329628 13721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:47.337793 13721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37677
I20260811 04:47:47.337826 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37677 every 8 connection(s)
I20260811 04:47:47.338866 13721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:47.340795 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13721
I20260811 04:47:47.340986 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:47.341318 23106 external_mini_cluster.cc:1428] Setting key ebe7bec4a92324d455fa9c90aff59ddd
I20260811 04:47:47.341922 13796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:47.348510 13796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e: Bootstrap starting.
I20260811 04:47:47.351003 13796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e: Neither blocks nor log segments found. Creating new log.
Aug 11 04:47:47 dist-test-slave-ss4n krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 04:47:47.351855 13796 log.cc:824] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:47.354087 13796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e: No bootstrap required, opened a new log
I20260811 04:47:47.357430 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:47.342950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:38903 (local address 127.0.0.1:37677)
0811 04:47:47.343438 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:47.343549 (+ 111us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:47.343590 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:47:47.344300 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:47.344318 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:47.344691 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:47.345157 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:47.346109 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:47.347243 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:47.348247 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:47.348859 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:47.349049 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:47.352092 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:47.352097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:47.352112 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:47.352145 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:47.354162 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:47.354803 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:47.354810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:47.354815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:47.354937 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:47.355329 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:47.355340 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:47.355342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:47.355771 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:47.356027 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:47.356381 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 04:47:47.356655 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":144,"threads_started":1}
I20260811 04:47:47.358006 13796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbb55396ed949429eb42a6e8b01f27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37677 } }
I20260811 04:47:47.358314 13796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:47.358366 13796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fbb55396ed949429eb42a6e8b01f27e, State: Initialized, Role: FOLLOWER
I20260811 04:47:47.358852 13796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbb55396ed949429eb42a6e8b01f27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37677 } }
I20260811 04:47:47.358989 13796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:47.359076 13796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:47.359191 13796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:47.360373 13796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbb55396ed949429eb42a6e8b01f27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37677 } }
I20260811 04:47:47.360749 13796 leader_election.cc:302] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fbb55396ed949429eb42a6e8b01f27e; no voters:
I20260811 04:47:47.361073 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:47.361176 13796 leader_election.cc:288] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:47.361255 13802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:47.361466 13802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [term 1 LEADER]: Becoming Leader. State: Replica: 2fbb55396ed949429eb42a6e8b01f27e, State: Running, Role: LEADER
I20260811 04:47:47.361897 13802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbb55396ed949429eb42a6e8b01f27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37677 } }
I20260811 04:47:47.362689 13796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:47.363982 13804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fbb55396ed949429eb42a6e8b01f27e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbb55396ed949429eb42a6e8b01f27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37677 } } }
I20260811 04:47:47.364145 13804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:47.363960 13803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fbb55396ed949429eb42a6e8b01f27e. Latest consensus state: current_term: 1 leader_uuid: "2fbb55396ed949429eb42a6e8b01f27e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbb55396ed949429eb42a6e8b01f27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37677 } } }
I20260811 04:47:47.364389 13803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:47.365952 13816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:47.369077 13816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:47.376003 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:47.364679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56005 (local address 127.0.0.1:37677)
0811 04:47:47.364798 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:47.364803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:47.365557 (+ 754us) server_negotiation.cc:407] Connection header received
0811 04:47:47.365738 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:47.365741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:47.365801 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:47.365885 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:47.367103 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:47.367885 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:47.369189 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:47.369559 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:47.369697 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:47.371717 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:47.371721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:47.371724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:47.371754 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:47.373662 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:47.374287 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:47.374291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:47.374293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:47.374392 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:47.374814 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:47.374818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:47.374820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:47.375048 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:47.375228 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:47.375756 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 04:47:47.375864 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 04:47:47.376169 13816 catalog_manager.cc:1347] Generated new cluster ID: 1a015efff67845d5bc7de17458e152db
I20260811 04:47:47.376394 13816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:47.393348 13816 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:47.394711 13816 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:47.408022 13816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fbb55396ed949429eb42a6e8b01f27e: Generated new TSK 0
I20260811 04:47:47.408922 13816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:47.413112 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13721
W20260811 04:47:47.423847 13706 connection.cc:535] client connection to 127.0.0.1:37677 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37677): Connection reset by peer (error 104)
2026-08-11T04:47:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:47 dist-test-slave-ss4n krb5kdc[13830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:47 dist-test-slave-ss4n krb5kdc[13830](info): set up 2 sockets
Aug 11 04:47:47 dist-test-slave-ss4n krb5kdc[13830](info): commencing operation
krb5kdc: starting...
W20260811 04:47:48.554656 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:48 dist-test-slave-ss4n krb5kdc[13830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:48.702198 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:48.828608 13845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:48.828946 13845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:48.829020 13845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:48.833159 13845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:48.833274 13845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:48.833313 13845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:48.833353 13845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:48.833385 13845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:48.839073 13845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:48.840384 13845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:48.841523 13845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:48.850582 13856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:48.850750 13845 server_base.cc:1034] running on GCE node
W20260811 04:47:48.850616 13851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:48.850620 13853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:48.851549 13845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:48.852756 13845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:48.853945 13845 hybrid_clock.cc:648] HybridClock initialized: now 1786423668853914 us; error 45 us; skew 500 ppm
Aug 11 04:47:48 dist-test-slave-ss4n krb5kdc[13830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:48.856829 13845 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:48.857808 13845 webserver.cc:460] Webserver started at http://127.0.0.1:39467/ using document root <none> and password file <none>
I20260811 04:47:48.858395 13845 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:48.858487 13845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:48.858726 13845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:48.860942 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "30c1a7518f264bc09209260e8237d335"
format_stamp: "Formatted at 2026-08-11 04:47:48 on dist-test-slave-ss4n"
server_key: "a61e32fa93c40666ac9d42c1588f06c8"
server_key_iv: "fdb22a73af014e99aa20014364f61165"
server_key_version: "encryptionkey@0"
I20260811 04:47:48.861590 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "30c1a7518f264bc09209260e8237d335"
format_stamp: "Formatted at 2026-08-11 04:47:48 on dist-test-slave-ss4n"
server_key: "a61e32fa93c40666ac9d42c1588f06c8"
server_key_iv: "fdb22a73af014e99aa20014364f61165"
server_key_version: "encryptionkey@0"
I20260811 04:47:48.865415 13845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:47:48.868131 13863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:48.869472 13845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 04:47:48.869704 13845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "30c1a7518f264bc09209260e8237d335"
format_stamp: "Formatted at 2026-08-11 04:47:48 on dist-test-slave-ss4n"
server_key: "a61e32fa93c40666ac9d42c1588f06c8"
server_key_iv: "fdb22a73af014e99aa20014364f61165"
server_key_version: "encryptionkey@0"
I20260811 04:47:48.869829 13845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:48.890137 13845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:48.893764 13845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:48.893930 13845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:48.901609 13845 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36823
I20260811 04:47:48.901618 13925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36823 every 8 connection(s)
I20260811 04:47:48.902717 13845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:48.905886 13926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:48.910919 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13845
I20260811 04:47:48.911046 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:48.911355 23106 external_mini_cluster.cc:1428] Setting key 8c3418d0b9ee2c4c86b768eb72a52ce2
I20260811 04:47:48.912086 13926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335: Bootstrap starting.
I20260811 04:47:48.914791 13926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:48.915684 13926 log.cc:824] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:48.918062 13926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335: No bootstrap required, opened a new log
I20260811 04:47:48.921788 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:48.912857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:44565 (local address 127.0.0.1:36823)
0811 04:47:48.913320 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:48.913337 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:48.913373 (+ 36us) server_negotiation.cc:407] Connection header received
0811 04:47:48.914124 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:48.914143 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:48.914511 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:48.914814 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:48.915235 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:48.916294 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:48.917288 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:48.918010 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:48.918199 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:48.919119 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:48.919126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:48.919143 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:48.919693 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:48.919765 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:48.919797 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:48.919937 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:48.920081 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 04:47:48.920227 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":165,"threads_started":1}
I20260811 04:47:48.922559 13926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c1a7518f264bc09209260e8237d335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260811 04:47:48.922840 13926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:48.922915 13926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c1a7518f264bc09209260e8237d335, State: Initialized, Role: FOLLOWER
I20260811 04:47:48.923357 13926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c1a7518f264bc09209260e8237d335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260811 04:47:48.923516 13926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:48.923609 13926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:48.923720 13926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:48.924913 13926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c1a7518f264bc09209260e8237d335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260811 04:47:48.925027 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:48.925305 13926 leader_election.cc:302] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30c1a7518f264bc09209260e8237d335; no voters:
I20260811 04:47:48.925676 13926 leader_election.cc:288] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:48.925800 13931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:48.926064 13931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [term 1 LEADER]: Becoming Leader. State: Replica: 30c1a7518f264bc09209260e8237d335, State: Running, Role: LEADER
I20260811 04:47:48.926424 13931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c1a7518f264bc09209260e8237d335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } }
I20260811 04:47:48.927045 13926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:48.927800 13931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c1a7518f264bc09209260e8237d335. Latest consensus state: current_term: 1 leader_uuid: "30c1a7518f264bc09209260e8237d335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c1a7518f264bc09209260e8237d335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } } }
I20260811 04:47:48.927932 13931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:48.928270 13932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30c1a7518f264bc09209260e8237d335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c1a7518f264bc09209260e8237d335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36823 } } }
I20260811 04:47:48.928357 13932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335 [sys.catalog]: This master's current role is: LEADER
W20260811 04:47:48.932518 13952 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:47:48.932605 13952 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:47:48.932703 13944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:48.932803 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:48.927462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51051 (local address 127.0.0.1:36823)
0811 04:47:48.927581 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:48.927585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:48.927975 (+ 390us) server_negotiation.cc:407] Connection header received
0811 04:47:48.928205 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:48.928208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:48.928274 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:48.928349 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:48.928763 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:48.929640 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:48.930738 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:48.931059 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:48.931173 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:48.932171 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:48.932175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:48.932178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:48.932192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:48.932220 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:48.932233 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:48.932328 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:48.932511 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 04:47:48.932554 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 04:47:48.933691 13944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:48.939953 13944 catalog_manager.cc:1347] Generated new cluster ID: af78858bcde44075ae66840586513eac
I20260811 04:47:48.940032 13944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:48.954322 13944 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:48.955641 13944 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:48.965749 13944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30c1a7518f264bc09209260e8237d335: Generated new TSK 0
I20260811 04:47:48.966557 13944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:48.979877 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13845
W20260811 04:47:48.988462 13828 connection.cc:535] client connection to 127.0.0.1:36823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36823): Connection reset by peer (error 104)
2026-08-11T04:47:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:49 dist-test-slave-ss4n krb5kdc[13959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:49 dist-test-slave-ss4n krb5kdc[13959](info): set up 2 sockets
Aug 11 04:47:49 dist-test-slave-ss4n krb5kdc[13959](info): commencing operation
krb5kdc: starting...
W20260811 04:47:50.127125 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:50 dist-test-slave-ss4n krb5kdc[13959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 04:47:50.273375 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:50.400408 13974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:50.400789 13974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:50.400878 13974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:50.405203 13974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:50.405303 13974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:50.405344 13974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:50.405402 13974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:50.405457 13974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:50.411019 13974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:50.412235 13974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:50.413339 13974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:50.422560 13981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:50.422668 13974 server_base.cc:1034] running on GCE node
W20260811 04:47:50.422559 13980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:50.422809 13983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:50.423511 13974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:50.424747 13974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:50.425976 13974 hybrid_clock.cc:648] HybridClock initialized: now 1786423670425956 us; error 35 us; skew 500 ppm
Aug 11 04:47:50 dist-test-slave-ss4n krb5kdc[13959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:50.429158 13974 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 04:47:50.430269 13974 webserver.cc:460] Webserver started at http://127.0.0.1:34899/ using document root <none> and password file <none>
I20260811 04:47:50.430859 13974 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:50.430955 13974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:50.431205 13974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:50.433408 13974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "52939aaea3b543e9b35b4c757fbf3583"
format_stamp: "Formatted at 2026-08-11 04:47:50 on dist-test-slave-ss4n"
server_key: "1558db1e4544cc9517795073292cb747"
server_key_iv: "c2f8250a621372d232f4a92fd8562fb1"
server_key_version: "encryptionkey@0"
I20260811 04:47:50.434084 13974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "52939aaea3b543e9b35b4c757fbf3583"
format_stamp: "Formatted at 2026-08-11 04:47:50 on dist-test-slave-ss4n"
server_key: "1558db1e4544cc9517795073292cb747"
server_key_iv: "c2f8250a621372d232f4a92fd8562fb1"
server_key_version: "encryptionkey@0"
I20260811 04:47:50.437996 13974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 04:47:50.440639 13991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:50.442051 13974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:50.442219 13974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "52939aaea3b543e9b35b4c757fbf3583"
format_stamp: "Formatted at 2026-08-11 04:47:50 on dist-test-slave-ss4n"
server_key: "1558db1e4544cc9517795073292cb747"
server_key_iv: "c2f8250a621372d232f4a92fd8562fb1"
server_key_version: "encryptionkey@0"
I20260811 04:47:50.442358 13974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:50.447505 13974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:50.451016 13974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:50.451239 13974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:50.459143 13974 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34293
I20260811 04:47:50.459218 14051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34293 every 8 connection(s)
I20260811 04:47:50.460347 13974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:50.462064 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 13974
I20260811 04:47:50.462267 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:50.462563 23106 external_mini_cluster.cc:1428] Setting key 3f72f1346f6ee6bf3d537a5903069d6d
I20260811 04:47:50.463769 14052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:50.468762 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:50.464105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:50795 (local address 127.0.0.1:34293)
0811 04:47:50.464515 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:50.464535 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:50.464577 (+ 42us) server_negotiation.cc:407] Connection header received
0811 04:47:50.465540 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:50.465565 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:50.465940 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:50.466339 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:50.467200 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:50.467206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:50.467223 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:50.467237 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:50.467307 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:50.467333 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:50.467525 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:50.467601 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 04:47:50.467772 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":119,"threads_started":1}
I20260811 04:47:50.471392 14052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583: Bootstrap starting.
I20260811 04:47:50.472124 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:50.474040 14052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:50.475075 14052 log.cc:824] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:50.476111 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:50.474331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:43823 (local address 127.0.0.1:34293)
0811 04:47:50.474482 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:50.474486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:50.474669 (+ 183us) server_negotiation.cc:407] Connection header received
0811 04:47:50.474796 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:50.474799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:50.474832 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:50.474898 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:50.475690 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:50.475693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:50.475696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:50.475710 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:50.475741 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:50.475758 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:50.475861 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:50.475907 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 04:47:50.475950 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 04:47:50.477459 14052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583: No bootstrap required, opened a new log
I20260811 04:47:50.481263 14052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52939aaea3b543e9b35b4c757fbf3583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 04:47:50.481560 14052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:50.481614 14052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52939aaea3b543e9b35b4c757fbf3583, State: Initialized, Role: FOLLOWER
I20260811 04:47:50.481999 14052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52939aaea3b543e9b35b4c757fbf3583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 04:47:50.482113 14052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:50.482162 14052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:50.482251 14052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:50.483178 14052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52939aaea3b543e9b35b4c757fbf3583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 04:47:50.483480 14052 leader_election.cc:302] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52939aaea3b543e9b35b4c757fbf3583; no voters:
I20260811 04:47:50.483763 14052 leader_election.cc:288] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:50.483906 14063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:50.484184 14063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [term 1 LEADER]: Becoming Leader. State: Replica: 52939aaea3b543e9b35b4c757fbf3583, State: Running, Role: LEADER
I20260811 04:47:50.484541 14063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52939aaea3b543e9b35b4c757fbf3583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } }
I20260811 04:47:50.485040 14052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:50.486469 14064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52939aaea3b543e9b35b4c757fbf3583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52939aaea3b543e9b35b4c757fbf3583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260811 04:47:50.486596 14064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:50.487138 14065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52939aaea3b543e9b35b4c757fbf3583. Latest consensus state: current_term: 1 leader_uuid: "52939aaea3b543e9b35b4c757fbf3583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52939aaea3b543e9b35b4c757fbf3583" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34293 } } }
I20260811 04:47:50.487257 14065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583 [sys.catalog]: This master's current role is: LEADER
W20260811 04:47:50.490003 14081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:47:50.490070 14081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:47:50.490159 14075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:50.490852 14075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:50.496310 14075 catalog_manager.cc:1347] Generated new cluster ID: 6d73dc5ab1fb44f8827c28423b03a9dc
I20260811 04:47:50.496379 14075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:50.514161 14075 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:50.514539 14075 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:50.526659 14075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52939aaea3b543e9b35b4c757fbf3583: Generated new TSK 0
I20260811 04:47:50.527501 14075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:50.531303 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 13974
2026-08-11T04:47:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:50 dist-test-slave-ss4n krb5kdc[14087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:50 dist-test-slave-ss4n krb5kdc[14087](info): set up 2 sockets
Aug 11 04:47:50 dist-test-slave-ss4n krb5kdc[14087](info): commencing operation
krb5kdc: starting...
W20260811 04:47:51.714592 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:51 dist-test-slave-ss4n krb5kdc[14087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:51.862316 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:46699
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:46699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:51.984272 14101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:51.984580 14101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:51.984624 14101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:51.988739 14101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:51.988822 14101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:51.988844 14101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:51.988862 14101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:51.988878 14101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:51.994532 14101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:46699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:46699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:51.995661 14101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:51.996686 14101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:52.006065 14110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:52.006114 14112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:52.006065 14109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:52.006352 14101 server_base.cc:1034] running on GCE node
I20260811 04:47:52.006963 14101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:52.008172 14101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:52.009341 14101 hybrid_clock.cc:648] HybridClock initialized: now 1786423672009321 us; error 34 us; skew 500 ppm
Aug 11 04:47:52 dist-test-slave-ss4n krb5kdc[14087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423672, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:52.012331 14101 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:52.013324 14101 webserver.cc:460] Webserver started at http://127.22.144.190:46017/ using document root <none> and password file <none>
I20260811 04:47:52.013985 14101 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:52.014088 14101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:52.014344 14101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:52.016566 14101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "efd421cfb8c449fea604a71afe98137f"
format_stamp: "Formatted at 2026-08-11 04:47:52 on dist-test-slave-ss4n"
server_key: "aedd472b47f8f1373ac36d5711e6b1b5"
server_key_iv: "78d376e4594d2bdd24bf6fa755697688"
server_key_version: "encryptionkey@0"
I20260811 04:47:52.017182 14101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "efd421cfb8c449fea604a71afe98137f"
format_stamp: "Formatted at 2026-08-11 04:47:52 on dist-test-slave-ss4n"
server_key: "aedd472b47f8f1373ac36d5711e6b1b5"
server_key_iv: "78d376e4594d2bdd24bf6fa755697688"
server_key_version: "encryptionkey@0"
I20260811 04:47:52.021412 14101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 04:47:52.024071 14119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:52.025586 14101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 04:47:52.025801 14101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "efd421cfb8c449fea604a71afe98137f"
format_stamp: "Formatted at 2026-08-11 04:47:52 on dist-test-slave-ss4n"
server_key: "aedd472b47f8f1373ac36d5711e6b1b5"
server_key_iv: "78d376e4594d2bdd24bf6fa755697688"
server_key_version: "encryptionkey@0"
I20260811 04:47:52.025941 14101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:52.058614 14101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:52.062469 14101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:52.062706 14101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:52.070767 14101 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:46699
I20260811 04:47:52.070798 14179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:46699 every 8 connection(s)
I20260811 04:47:52.071851 14101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:52.074934 14181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:52.080943 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14101
I20260811 04:47:52.081074 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:52.081385 23106 external_mini_cluster.cc:1428] Setting key 84f76d016dd2db1d10e9477d3bcc9b9f
I20260811 04:47:52.081530 14181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f: Bootstrap starting.
I20260811 04:47:52.084623 14181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:52.085664 14181 log.cc:824] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:52.088250 14181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f: No bootstrap required, opened a new log
Aug 11 04:47:52 dist-test-slave-ss4n krb5kdc[14087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:52.092762 14181 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd421cfb8c449fea604a71afe98137f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 46699 } }
I20260811 04:47:52.093042 14181 raft_consensus.cc:374] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:52.093111 14181 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd421cfb8c449fea604a71afe98137f, State: Initialized, Role: FOLLOWER
I20260811 04:47:52.093684 14181 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd421cfb8c449fea604a71afe98137f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 46699 } }
I20260811 04:47:52.093911 14181 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:52.093995 14181 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:52.094115 14181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:52.095165 14181 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd421cfb8c449fea604a71afe98137f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 46699 } }
I20260811 04:47:52.095563 14181 leader_election.cc:302] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd421cfb8c449fea604a71afe98137f; no voters:
I20260811 04:47:52.095978 14181 leader_election.cc:288] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:52.096062 14187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:52.096292 14187 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [term 1 LEADER]: Becoming Leader. State: Replica: efd421cfb8c449fea604a71afe98137f, State: Running, Role: LEADER
I20260811 04:47:52.096618 14187 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd421cfb8c449fea604a71afe98137f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 46699 } }
I20260811 04:47:52.097430 14181 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:52.097909 14189 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd421cfb8c449fea604a71afe98137f. Latest consensus state: current_term: 1 leader_uuid: "efd421cfb8c449fea604a71afe98137f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd421cfb8c449fea604a71afe98137f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 46699 } } }
I20260811 04:47:52.098039 14189 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:52.098294 14188 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efd421cfb8c449fea604a71afe98137f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd421cfb8c449fea604a71afe98137f" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 46699 } } }
I20260811 04:47:52.098397 14188 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:52.098835 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:52.083005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52907 (local address 127.22.144.190:46699)
0811 04:47:52.083529 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:52.083551 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:52.083595 (+ 44us) server_negotiation.cc:407] Connection header received
0811 04:47:52.084529 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:52.084559 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:52.085062 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:52.085491 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:52.086432 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:52.087975 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:52.089216 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:52.089974 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:52.090171 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:52.093168 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:52.093176 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:52.093196 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:52.093232 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:52.095166 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:52.095756 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:52.095762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:52.095768 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:52.095901 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:52.096304 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:52.096307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:52.096309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:52.096881 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:52.097108 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:52.097507 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 04:47:52.097778 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":149,"threads_started":1}
I20260811 04:47:52.098835 14191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:52.102918 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:52.103562 14191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:52.110391 14191 catalog_manager.cc:1347] Generated new cluster ID: 007a38a5924f455fad8d43cb5fc2725e
I20260811 04:47:52.110472 14191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:52.116472 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:52.106133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44725 (local address 127.22.144.190:46699)
0811 04:47:52.106286 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:52.106290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:52.106694 (+ 404us) server_negotiation.cc:407] Connection header received
0811 04:47:52.106803 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:52.106808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:52.106863 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:52.106955 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:52.107894 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:52.108817 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:52.110101 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:52.110521 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:52.110605 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:52.112644 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:52.112648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:52.112651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:52.112678 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:52.114437 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:52.115041 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:52.115044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:52.115046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:52.115135 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:52.115505 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:52.115507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:52.115509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:52.115732 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:52.115878 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:52.116200 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 04:47:52.116306 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 04:47:52.125571 14191 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:52.126924 14191 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:52.134941 14191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efd421cfb8c449fea604a71afe98137f: Generated new TSK 0
I20260811 04:47:52.135854 14191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:52.141583 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14101
W20260811 04:47:52.150835 14085 connection.cc:535] client connection to 127.22.144.190:46699 recv error: Network error: failed to read from TLS socket (remote: 127.22.144.190:46699): Connection reset by peer (error 104)
2026-08-11T04:47:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1611 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:52 dist-test-slave-ss4n krb5kdc[14219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:52 dist-test-slave-ss4n krb5kdc[14219](info): set up 2 sockets
Aug 11 04:47:52 dist-test-slave-ss4n krb5kdc[14219](info): commencing operation
krb5kdc: starting...
W20260811 04:47:53.276958 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:53 dist-test-slave-ss4n krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:53.430676 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44807
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:35511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:53.549104 14234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:53.549418 14234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:53.549463 14234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:53.553682 14234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:53.553764 14234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:53.553784 14234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:53.553802 14234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:53.553817 14234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:53.559476 14234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:35511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:44807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:53.560740 14234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:53.561913 14234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:53.571497 14240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:53.571480 14239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:53.571480 14242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:53.572058 14234 server_base.cc:1034] running on GCE node
I20260811 04:47:53.572671 14234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:53.574012 14234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:53.575196 14234 hybrid_clock.cc:648] HybridClock initialized: now 1786423673575104 us; error 103 us; skew 500 ppm
Aug 11 04:47:53 dist-test-slave-ss4n krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423673, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:53.583636 14234 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:53.584808 14234 webserver.cc:460] Webserver started at http://127.22.144.190:33537/ using document root <none> and password file <none>
I20260811 04:47:53.585577 14234 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:53.585655 14234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:53.585923 14234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:53.588282 14234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "50fa0266c85048dca99f0bcc2d2d7c54"
format_stamp: "Formatted at 2026-08-11 04:47:53 on dist-test-slave-ss4n"
server_key: "69ed47ef71c5f8b97eabdd5391d6d8d0"
server_key_iv: "0d8ace37585422ac4c7c3b5d0dd332d5"
server_key_version: "encryptionkey@0"
I20260811 04:47:53.588943 14234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "50fa0266c85048dca99f0bcc2d2d7c54"
format_stamp: "Formatted at 2026-08-11 04:47:53 on dist-test-slave-ss4n"
server_key: "69ed47ef71c5f8b97eabdd5391d6d8d0"
server_key_iv: "0d8ace37585422ac4c7c3b5d0dd332d5"
server_key_version: "encryptionkey@0"
I20260811 04:47:53.593310 14234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 04:47:53.596168 14249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:53.597584 14234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:53.597734 14234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "50fa0266c85048dca99f0bcc2d2d7c54"
format_stamp: "Formatted at 2026-08-11 04:47:53 on dist-test-slave-ss4n"
server_key: "69ed47ef71c5f8b97eabdd5391d6d8d0"
server_key_iv: "0d8ace37585422ac4c7c3b5d0dd332d5"
server_key_version: "encryptionkey@0"
I20260811 04:47:53.597869 14234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:53.624517 14234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:53.628307 14234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:53.628556 14234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:53.637624 14234 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:44807
I20260811 04:47:53.637625 14307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:44807 every 8 connection(s)
I20260811 04:47:53.638768 14234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:53.639307 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14234
I20260811 04:47:53.639458 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:53.639802 23106 external_mini_cluster.cc:1428] Setting key 43c76dc55befd2935481f779bbfcf2fa
I20260811 04:47:53.642527 14309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:53.649808 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:53.641385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36673 (local address 127.22.144.190:44807)
0811 04:47:53.641908 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:53.641936 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:53.641973 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:47:53.642675 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:53.642699 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:53.643071 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:53.643373 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:53.643803 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:53.644818 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:53.646233 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:53.646735 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:53.646864 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:53.647920 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:53.647925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:53.647938 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:53.648334 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:53.648393 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:53.648417 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:53.648521 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:53.648783 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 04:47:53.648960 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":202,"threads_started":1}
I20260811 04:47:53.649960 14309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54: Bootstrap starting.
I20260811 04:47:53.652859 14309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:53.653214 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:53.653949 14309 log.cc:824] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:53.656394 14309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54: No bootstrap required, opened a new log
I20260811 04:47:53.660318 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:53.655777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41919 (local address 127.22.144.190:44807)
0811 04:47:53.655923 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:53.655928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:53.656118 (+ 190us) server_negotiation.cc:407] Connection header received
0811 04:47:53.656274 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:53.656276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:53.656316 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:53.656386 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:53.656687 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:53.657247 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:53.658468 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:53.658743 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:53.658835 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:53.659787 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:53.659790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:53.659793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:53.659805 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:53.659832 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:53.659843 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:53.659920 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:53.660095 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 04:47:53.660156 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 04:47:53.661173 14309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44807 } }
I20260811 04:47:53.661541 14309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:53.661643 14309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50fa0266c85048dca99f0bcc2d2d7c54, State: Initialized, Role: FOLLOWER
I20260811 04:47:53.662138 14309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44807 } }
I20260811 04:47:53.662288 14309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:53.662371 14309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:53.662499 14309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:53.663522 14309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44807 } }
I20260811 04:47:53.663903 14309 leader_election.cc:302] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50fa0266c85048dca99f0bcc2d2d7c54; no voters:
I20260811 04:47:53.664281 14309 leader_election.cc:288] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:53.664357 14320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:53.664598 14320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [term 1 LEADER]: Becoming Leader. State: Replica: 50fa0266c85048dca99f0bcc2d2d7c54, State: Running, Role: LEADER
I20260811 04:47:53.664975 14320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44807 } }
I20260811 04:47:53.665809 14309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:53.666857 14322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44807 } } }
I20260811 04:47:53.666999 14322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:53.667424 14321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50fa0266c85048dca99f0bcc2d2d7c54. Latest consensus state: current_term: 1 leader_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50fa0266c85048dca99f0bcc2d2d7c54" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 44807 } } }
I20260811 04:47:53.667613 14321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:53.667961 14329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:53.671240 14329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:53.679133 14329 catalog_manager.cc:1347] Generated new cluster ID: 00917d1d0dce4cf487552af2fcbfb4e5
I20260811 04:47:53.679239 14329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:53.698220 14329 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:53.699319 14329 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:53.709636 14329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50fa0266c85048dca99f0bcc2d2d7c54: Generated new TSK 0
I20260811 04:47:53.710618 14329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:53.727556 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14234
W20260811 04:47:53.737720 14215 connection.cc:535] client connection to 127.22.144.190:44807 recv error: Network error: failed to read from TLS socket (remote: 127.22.144.190:44807): Connection reset by peer (error 104)
2026-08-11T04:47:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:53 dist-test-slave-ss4n krb5kdc[14344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:53 dist-test-slave-ss4n krb5kdc[14344](info): set up 2 sockets
Aug 11 04:47:53 dist-test-slave-ss4n krb5kdc[14344](info): commencing operation
krb5kdc: starting...
W20260811 04:47:54.875900 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:54 dist-test-slave-ss4n krb5kdc[14344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:55.017215 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:38021
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:37113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:38021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:55.136798 14358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:55.137122 14358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:55.137176 14358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:55.141377 14358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:55.141463 14358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:55.141507 14358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:55.141525 14358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:55.141541 14358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:55.147234 14358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:37113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:38021
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:55.148413 14358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:55.149546 14358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:55.159325 14363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:55.159337 14366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:55.159392 14358 server_base.cc:1034] running on GCE node
W20260811 04:47:55.159339 14364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:55.160356 14358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:55.161751 14358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:55.162954 14358 hybrid_clock.cc:648] HybridClock initialized: now 1786423675162943 us; error 35 us; skew 500 ppm
Aug 11 04:47:55 dist-test-slave-ss4n krb5kdc[14344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423675, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:55.166222 14358 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:55.167315 14358 webserver.cc:460] Webserver started at http://127.22.144.190:36575/ using document root <none> and password file <none>
I20260811 04:47:55.167933 14358 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:55.168002 14358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:55.168274 14358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:55.170672 14358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "a3c41d0b5a854840a9aa9bdc102a921a"
format_stamp: "Formatted at 2026-08-11 04:47:55 on dist-test-slave-ss4n"
server_key: "504c0579958384dc7031a22597d93550"
server_key_iv: "f251133a9b3506157850d4796537924c"
server_key_version: "encryptionkey@0"
I20260811 04:47:55.171348 14358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "a3c41d0b5a854840a9aa9bdc102a921a"
format_stamp: "Formatted at 2026-08-11 04:47:55 on dist-test-slave-ss4n"
server_key: "504c0579958384dc7031a22597d93550"
server_key_iv: "f251133a9b3506157850d4796537924c"
server_key_version: "encryptionkey@0"
I20260811 04:47:55.175712 14358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:55.178596 14375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:55.179991 14358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 04:47:55.180159 14358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "a3c41d0b5a854840a9aa9bdc102a921a"
format_stamp: "Formatted at 2026-08-11 04:47:55 on dist-test-slave-ss4n"
server_key: "504c0579958384dc7031a22597d93550"
server_key_iv: "f251133a9b3506157850d4796537924c"
server_key_version: "encryptionkey@0"
I20260811 04:47:55.180290 14358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:55.185649 14358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:55.189075 14358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:55.189256 14358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:55.196779 14358 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:38021
I20260811 04:47:55.196807 14430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:38021 every 8 connection(s)
I20260811 04:47:55.197881 14358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:55.200892 14431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:55.205381 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14358
I20260811 04:47:55.205591 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:55.205919 23106 external_mini_cluster.cc:1428] Setting key 7a662f53bfa9aef65a1b880fbdf31f7a
I20260811 04:47:55.207724 14431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a: Bootstrap starting.
I20260811 04:47:55.210390 14431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:55.211300 14431 log.cc:824] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:55.211838 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:55.207518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35689 (local address 127.22.144.190:38021)
0811 04:47:55.208053 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:55.208069 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:55.208108 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:47:55.208813 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:55.208831 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:55.209149 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:55.209546 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:55.210425 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:55.210430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:55.210464 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:55.210480 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:55.210558 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:55.210582 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:55.210714 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:55.210818 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 04:47:55.210987 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":161,"threads_started":1}
I20260811 04:47:55.213349 14431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a: No bootstrap required, opened a new log
I20260811 04:47:55.214708 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:55.217844 14431 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38021 } }
I20260811 04:47:55.218171 14431 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:55.218247 14431 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c41d0b5a854840a9aa9bdc102a921a, State: Initialized, Role: FOLLOWER
I20260811 04:47:55.218806 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:55.216855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52255 (local address 127.22.144.190:38021)
0811 04:47:55.216974 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:55.216978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:55.217219 (+ 241us) server_negotiation.cc:407] Connection header received
0811 04:47:55.217360 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:55.217363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:55.217394 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:55.217452 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:55.218278 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:55.218282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:55.218285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:55.218300 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:55.218332 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:55.218346 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:55.218406 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:55.218606 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 04:47:55.218649 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 04:47:55.218768 14431 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38021 } }
I20260811 04:47:55.218943 14431 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:55.219029 14431 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:55.219130 14431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:55.220260 14431 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38021 } }
I20260811 04:47:55.220614 14431 leader_election.cc:302] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c41d0b5a854840a9aa9bdc102a921a; no voters:
I20260811 04:47:55.220924 14431 leader_election.cc:288] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:55.221042 14442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:55.221315 14442 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [term 1 LEADER]: Becoming Leader. State: Replica: a3c41d0b5a854840a9aa9bdc102a921a, State: Running, Role: LEADER
I20260811 04:47:55.221771 14442 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38021 } }
I20260811 04:47:55.222316 14431 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:55.223552 14443 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38021 } } }
I20260811 04:47:55.223690 14443 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:55.224069 14444 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c41d0b5a854840a9aa9bdc102a921a. Latest consensus state: current_term: 1 leader_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c41d0b5a854840a9aa9bdc102a921a" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 38021 } } }
I20260811 04:47:55.224160 14453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:55.224231 14444 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:55.227746 14453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:55.233741 14453 catalog_manager.cc:1347] Generated new cluster ID: ac7ff055e21d44859306b4aa60fb8dd8
I20260811 04:47:55.233820 14453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:55.262545 14453 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:55.262959 14453 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:55.270500 14453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3c41d0b5a854840a9aa9bdc102a921a: Generated new TSK 0
I20260811 04:47:55.271286 14453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:55.276355 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14358
2026-08-11T04:47:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:55 dist-test-slave-ss4n krb5kdc[14465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:55 dist-test-slave-ss4n krb5kdc[14465](info): set up 2 sockets
Aug 11 04:47:55 dist-test-slave-ss4n krb5kdc[14465](info): commencing operation
krb5kdc: starting...
W20260811 04:47:56.434716 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:56 dist-test-slave-ss4n krb5kdc[14465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:56.580654 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35239
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:42225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:35239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:56.700893 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:56.701213 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:56.701263 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:56.705818 14480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:56.705896 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:56.705919 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:56.705937 14480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:56.705955 14480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:56.711823 14480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:42225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:35239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:35239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:56.712970 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:56.714133 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:56.723438 14487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:56.723527 14490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:56.723439 14488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:56.723874 14480 server_base.cc:1034] running on GCE node
I20260811 04:47:56.724784 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:56.726245 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:56.727448 14480 hybrid_clock.cc:648] HybridClock initialized: now 1786423676727423 us; error 59 us; skew 500 ppm
Aug 11 04:47:56 dist-test-slave-ss4n krb5kdc[14465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423676, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:56.731038 14480 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:56.732300 14480 webserver.cc:460] Webserver started at http://127.22.144.190:36491/ using document root <none> and password file <none>
I20260811 04:47:56.733031 14480 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:56.733147 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:56.733443 14480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:56.735972 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "51ab0d7099e3414ea87dc848c5c13990"
format_stamp: "Formatted at 2026-08-11 04:47:56 on dist-test-slave-ss4n"
server_key: "57016dace8dc0e809ac620c258b3db7a"
server_key_iv: "4bb33e2d81ff10136dfef47e713859cb"
server_key_version: "encryptionkey@0"
I20260811 04:47:56.736696 14480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "51ab0d7099e3414ea87dc848c5c13990"
format_stamp: "Formatted at 2026-08-11 04:47:56 on dist-test-slave-ss4n"
server_key: "57016dace8dc0e809ac620c258b3db7a"
server_key_iv: "4bb33e2d81ff10136dfef47e713859cb"
server_key_version: "encryptionkey@0"
I20260811 04:47:56.741295 14480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 04:47:56.744231 14497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:56.745595 14480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:56.745761 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "51ab0d7099e3414ea87dc848c5c13990"
format_stamp: "Formatted at 2026-08-11 04:47:56 on dist-test-slave-ss4n"
server_key: "57016dace8dc0e809ac620c258b3db7a"
server_key_iv: "4bb33e2d81ff10136dfef47e713859cb"
server_key_version: "encryptionkey@0"
I20260811 04:47:56.745898 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:56.759796 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:56.763890 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:56.764173 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:56.772639 14480 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:35239
I20260811 04:47:56.772670 14557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:35239 every 8 connection(s)
I20260811 04:47:56.773830 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:56.776934 14558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:56.779239 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14480
I20260811 04:47:56.779417 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:56.779774 23106 external_mini_cluster.cc:1428] Setting key 7d2b4786c2f624aab0ec0ae87299f150
I20260811 04:47:56.784788 14558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990: Bootstrap starting.
I20260811 04:47:56.787470 14558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:56.788690 14558 log.cc:824] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:47:56 dist-test-slave-ss4n krb5kdc[14465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:47:56.791538 14558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990: No bootstrap required, opened a new log
I20260811 04:47:56.796882 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:56.781386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38279 (local address 127.22.144.190:35239)
0811 04:47:56.781912 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:56.781928 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:56.781966 (+ 38us) server_negotiation.cc:407] Connection header received
0811 04:47:56.782664 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:56.782682 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:56.783141 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:56.783486 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:56.784647 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:56.785786 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:56.786979 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:56.787469 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:56.787829 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:56.790741 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:56.790749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:56.790768 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:56.790805 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:56.793089 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:56.793687 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:56.793694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:56.793701 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:56.793862 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:56.794190 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:56.794195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:56.794199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:56.794886 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:56.795119 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:56.795416 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 04:47:56.795760 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":162,"threads_started":1}
I20260811 04:47:56.797037 14558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ab0d7099e3414ea87dc848c5c13990" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35239 } }
I20260811 04:47:56.797351 14558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:56.797417 14558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ab0d7099e3414ea87dc848c5c13990, State: Initialized, Role: FOLLOWER
I20260811 04:47:56.797984 14558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ab0d7099e3414ea87dc848c5c13990" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35239 } }
I20260811 04:47:56.798170 14558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:56.798249 14558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:56.798386 14558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:56.799701 14558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ab0d7099e3414ea87dc848c5c13990" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35239 } }
I20260811 04:47:56.800133 14558 leader_election.cc:302] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51ab0d7099e3414ea87dc848c5c13990; no voters:
I20260811 04:47:56.800555 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:56.800577 14558 leader_election.cc:288] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:56.800659 14563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:56.800879 14563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [term 1 LEADER]: Becoming Leader. State: Replica: 51ab0d7099e3414ea87dc848c5c13990, State: Running, Role: LEADER
I20260811 04:47:56.801290 14563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ab0d7099e3414ea87dc848c5c13990" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35239 } }
I20260811 04:47:56.802048 14558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:56.802744 14563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ab0d7099e3414ea87dc848c5c13990. Latest consensus state: current_term: 1 leader_uuid: "51ab0d7099e3414ea87dc848c5c13990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ab0d7099e3414ea87dc848c5c13990" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35239 } } }
I20260811 04:47:56.802922 14563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:56.803339 14564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51ab0d7099e3414ea87dc848c5c13990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ab0d7099e3414ea87dc848c5c13990" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 35239 } } }
I20260811 04:47:56.803472 14564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:56.803812 14574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:56.807457 14574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:56.813313 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:56.803484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40505 (local address 127.22.144.190:35239)
0811 04:47:56.803620 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:56.803628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:56.803909 (+ 281us) server_negotiation.cc:407] Connection header received
0811 04:47:56.804148 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:56.804152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:56.804204 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:56.804260 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:56.805358 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:56.806172 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:56.807361 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:56.807730 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:56.807844 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:56.809923 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:56.809927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:56.809929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:47:56.809955 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:56.811490 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:56.811997 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:56.812000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:56.812002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:56.812097 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:47:56.812407 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:47:56.812410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:47:56.812412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:47:56.812618 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:56.812759 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:56.813030 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 04:47:56.813156 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 04:47:56.815217 14574 catalog_manager.cc:1347] Generated new cluster ID: 4381ab3dac1040299f03013f96f39b7f
I20260811 04:47:56.815354 14574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:56.837764 14574 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:56.839241 14574 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:56.845660 14574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51ab0d7099e3414ea87dc848c5c13990: Generated new TSK 0
I20260811 04:47:56.846464 14574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:56.851920 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14480
2026-08-11T04:47:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:56 dist-test-slave-ss4n krb5kdc[14590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:56 dist-test-slave-ss4n krb5kdc[14590](info): set up 2 sockets
Aug 11 04:47:56 dist-test-slave-ss4n krb5kdc[14590](info): commencing operation
krb5kdc: starting...
W20260811 04:47:58.003031 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:58 dist-test-slave-ss4n krb5kdc[14590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:58.149832 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:33121
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:40243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:33121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:58.269146 14606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:58.269562 14606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:58.269658 14606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:58.274009 14606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:58.274137 14606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:58.274171 14606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:58.274204 14606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:58.274262 14606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:58.279912 14606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:40243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:33121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:33121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:58.281131 14606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:58.282284 14606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:58.291590 14613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:58.291610 14615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:58.291597 14612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:58.292037 14606 server_base.cc:1034] running on GCE node
I20260811 04:47:58.292675 14606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:58.293947 14606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:58.295145 14606 hybrid_clock.cc:648] HybridClock initialized: now 1786423678295132 us; error 29 us; skew 500 ppm
Aug 11 04:47:58 dist-test-slave-ss4n krb5kdc[14590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423678, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:58.298787 14606 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:58.299855 14606 webserver.cc:460] Webserver started at http://127.22.144.190:33749/ using document root <none> and password file <none>
I20260811 04:47:58.300462 14606 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:58.300527 14606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:58.300803 14606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:58.303078 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "4a9b54503d7b442a8d242b205731d753"
format_stamp: "Formatted at 2026-08-11 04:47:58 on dist-test-slave-ss4n"
server_key: "08e696ba56f83fd4de2fe9380d6be304"
server_key_iv: "7f9cb5c03cc7ce95d416836518ffd7d4"
server_key_version: "encryptionkey@0"
I20260811 04:47:58.303714 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "4a9b54503d7b442a8d242b205731d753"
format_stamp: "Formatted at 2026-08-11 04:47:58 on dist-test-slave-ss4n"
server_key: "08e696ba56f83fd4de2fe9380d6be304"
server_key_iv: "7f9cb5c03cc7ce95d416836518ffd7d4"
server_key_version: "encryptionkey@0"
I20260811 04:47:58.307899 14606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:47:58.310595 14623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:58.311977 14606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:47:58.312122 14606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "4a9b54503d7b442a8d242b205731d753"
format_stamp: "Formatted at 2026-08-11 04:47:58 on dist-test-slave-ss4n"
server_key: "08e696ba56f83fd4de2fe9380d6be304"
server_key_iv: "7f9cb5c03cc7ce95d416836518ffd7d4"
server_key_version: "encryptionkey@0"
I20260811 04:47:58.312248 14606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:58.331873 14606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:58.335433 14606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:58.335681 14606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:58.344277 14606 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:33121
I20260811 04:47:58.344386 14680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:33121 every 8 connection(s)
I20260811 04:47:58.345369 14606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:58.348258 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14606
I20260811 04:47:58.348387 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:58.348691 14681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:58.348711 23106 external_mini_cluster.cc:1428] Setting key 22ccbc907cd215fef405c3122741c92e
I20260811 04:47:58.356957 14681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753: Bootstrap starting.
I20260811 04:47:58.359217 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:58.350305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47877 (local address 127.22.144.190:33121)
0811 04:47:58.350744 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:58.350761 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:58.350798 (+ 37us) server_negotiation.cc:407] Connection header received
0811 04:47:58.351793 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:58.351813 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:58.352176 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:58.352466 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:58.352905 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:58.354081 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:58.355480 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:58.356001 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:58.356139 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:58.357215 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:58.357221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:58.357235 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:58.357780 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:58.357841 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:58.357866 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:58.357949 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:58.358144 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 04:47:58.358386 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":149,"threads_started":1}
I20260811 04:47:58.359823 14681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:58.360623 14681 log.cc:824] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:58.362383 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:58.363047 14681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753: No bootstrap required, opened a new log
I20260811 04:47:58.366976 14681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9b54503d7b442a8d242b205731d753" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33121 } }
I20260811 04:47:58.367303 14681 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:58.367381 14681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a9b54503d7b442a8d242b205731d753, State: Initialized, Role: FOLLOWER
I20260811 04:47:58.367924 14681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9b54503d7b442a8d242b205731d753" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33121 } }
I20260811 04:47:58.368099 14681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:58.368207 14681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:58.368355 14681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:58.369248 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:58.364589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47267 (local address 127.22.144.190:33121)
0811 04:47:58.364728 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:58.364733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:58.364899 (+ 166us) server_negotiation.cc:407] Connection header received
0811 04:47:58.365058 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:58.365061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:58.365115 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:58.365209 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:58.365550 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:58.366455 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:58.367367 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:47:58.367725 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:47:58.367830 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:47:58.368803 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:58.368807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:58.368809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:58.368822 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:58.368857 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:58.368868 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:58.368946 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:58.369047 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 04:47:58.369103 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 04:47:58.369668 14681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9b54503d7b442a8d242b205731d753" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33121 } }
I20260811 04:47:58.370074 14681 leader_election.cc:302] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a9b54503d7b442a8d242b205731d753; no voters:
I20260811 04:47:58.370514 14681 leader_election.cc:288] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:58.370590 14692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:58.370826 14692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [term 1 LEADER]: Becoming Leader. State: Replica: 4a9b54503d7b442a8d242b205731d753, State: Running, Role: LEADER
I20260811 04:47:58.371203 14692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9b54503d7b442a8d242b205731d753" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33121 } }
I20260811 04:47:58.372046 14681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:58.373222 14696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a9b54503d7b442a8d242b205731d753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9b54503d7b442a8d242b205731d753" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33121 } } }
I20260811 04:47:58.373369 14696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:58.373795 14693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a9b54503d7b442a8d242b205731d753. Latest consensus state: current_term: 1 leader_uuid: "4a9b54503d7b442a8d242b205731d753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9b54503d7b442a8d242b205731d753" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 33121 } } }
I20260811 04:47:58.373907 14693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:58.374643 14703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:58.377655 14703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:58.384207 14703 catalog_manager.cc:1347] Generated new cluster ID: 6dae8dc2246e4ca38158b1d78b957506
I20260811 04:47:58.384303 14703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:58.408740 14703 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:58.410207 14703 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:58.416790 14703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a9b54503d7b442a8d242b205731d753: Generated new TSK 0
I20260811 04:47:58.417762 14703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:58.429198 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14606
2026-08-11T04:47:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:47:58 dist-test-slave-ss4n krb5kdc[14715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:47:58 dist-test-slave-ss4n krb5kdc[14715](info): set up 2 sockets
Aug 11 04:47:58 dist-test-slave-ss4n krb5kdc[14715](info): commencing operation
krb5kdc: starting...
W20260811 04:47:59.604665 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:47:59 dist-test-slave-ss4n krb5kdc[14715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:47:59.749874 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41065
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:40261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:41065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:47:59.868129 14728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:47:59.868454 14728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:47:59.868539 14728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:47:59.872676 14728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:47:59.872781 14728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:47:59.872820 14728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:47:59.872881 14728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:47:59.872921 14728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:47:59.878460 14728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:40261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:41065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:41065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:47:59.879580 14728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:47:59.880658 14728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:47:59.889729 14738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:59.889739 14735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:47:59.889747 14734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:47:59.889796 14728 server_base.cc:1034] running on GCE node
I20260811 04:47:59.890647 14728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:47:59.891819 14728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:47:59.893019 14728 hybrid_clock.cc:648] HybridClock initialized: now 1786423679893003 us; error 38 us; skew 500 ppm
Aug 11 04:47:59 dist-test-slave-ss4n krb5kdc[14715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423679, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:47:59.896075 14728 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:47:59.897095 14728 webserver.cc:460] Webserver started at http://127.22.144.190:42185/ using document root <none> and password file <none>
I20260811 04:47:59.897790 14728 fs_manager.cc:359] Metadata directory not provided
I20260811 04:47:59.897859 14728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:47:59.898109 14728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:47:59.900240 14728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "19b62c84b17c45fc9fddd075d86ca1b0"
format_stamp: "Formatted at 2026-08-11 04:47:59 on dist-test-slave-ss4n"
server_key: "700849eaf3a2f5c2a46b3d3a32e3190b"
server_key_iv: "ead0d35a4e3b899ae48d504457f2977b"
server_key_version: "encryptionkey@0"
I20260811 04:47:59.900852 14728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "19b62c84b17c45fc9fddd075d86ca1b0"
format_stamp: "Formatted at 2026-08-11 04:47:59 on dist-test-slave-ss4n"
server_key: "700849eaf3a2f5c2a46b3d3a32e3190b"
server_key_iv: "ead0d35a4e3b899ae48d504457f2977b"
server_key_version: "encryptionkey@0"
I20260811 04:47:59.904815 14728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:47:59.907390 14745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:47:59.908748 14728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:47:59.908882 14728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "19b62c84b17c45fc9fddd075d86ca1b0"
format_stamp: "Formatted at 2026-08-11 04:47:59 on dist-test-slave-ss4n"
server_key: "700849eaf3a2f5c2a46b3d3a32e3190b"
server_key_iv: "ead0d35a4e3b899ae48d504457f2977b"
server_key_version: "encryptionkey@0"
I20260811 04:47:59.908993 14728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:47:59.913919 14728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:47:59.918649 14728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:47:59.918864 14728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:47:59.927518 14728 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:41065
I20260811 04:47:59.927587 14806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:41065 every 8 connection(s)
I20260811 04:47:59.928539 14728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:47:59.931643 14807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:47:59.937431 14807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0: Bootstrap starting.
I20260811 04:47:59.937911 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14728
I20260811 04:47:59.938071 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:47:59.938375 23106 external_mini_cluster.cc:1428] Setting key 5a2263c0d988dfe88e41171018c93321
I20260811 04:47:59.940310 14807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0: Neither blocks nor log segments found. Creating new log.
I20260811 04:47:59.941114 14807 log.cc:824] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0: Log is configured to *not* fsync() on all Append() calls
I20260811 04:47:59.943639 14807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0: No bootstrap required, opened a new log
I20260811 04:47:59.944056 14811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:59.939818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43077 (local address 127.22.144.190:41065)
0811 04:47:59.940316 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:59.940331 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:59.940392 (+ 61us) server_negotiation.cc:407] Connection header received
0811 04:47:59.941082 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:59.941107 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:59.941453 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:59.941926 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:59.942746 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:59.942751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:59.942764 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:59.942773 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:59.942823 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:59.942841 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:59.942931 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:59.943080 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 04:47:59.943288 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":168,"threads_started":1}
I20260811 04:47:59.946982 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:47:59.947676 14807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41065 } }
I20260811 04:47:59.947904 14807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 04:47:59.947968 14807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b62c84b17c45fc9fddd075d86ca1b0, State: Initialized, Role: FOLLOWER
I20260811 04:47:59.948489 14807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41065 } }
I20260811 04:47:59.948653 14807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 04:47:59.948740 14807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 04:47:59.948849 14807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:47:59.950240 14807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41065 } }
I20260811 04:47:59.950634 14807 leader_election.cc:302] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19b62c84b17c45fc9fddd075d86ca1b0; no voters:
I20260811 04:47:59.951007 14807 leader_election.cc:288] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 04:47:59.951084 14818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:47:59.951220 14811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:47:59.949303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53603 (local address 127.22.144.190:41065)
0811 04:47:59.949443 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:47:59.949448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:47:59.949673 (+ 225us) server_negotiation.cc:407] Connection header received
0811 04:47:59.949782 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:47:59.949785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:47:59.949816 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:47:59.949883 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:47:59.950715 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:47:59.950718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:47:59.950721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 04:47:59.950733 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:47:59.950771 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 04:47:59.950803 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:47:59.950882 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 04:47:59.950975 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 04:47:59.951020 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 04:47:59.951347 14818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [term 1 LEADER]: Becoming Leader. State: Replica: 19b62c84b17c45fc9fddd075d86ca1b0, State: Running, Role: LEADER
I20260811 04:47:59.951694 14818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41065 } }
I20260811 04:47:59.952528 14807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:47:59.953680 14820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19b62c84b17c45fc9fddd075d86ca1b0. Latest consensus state: current_term: 1 leader_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41065 } } }
I20260811 04:47:59.953827 14820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:59.954205 14819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b62c84b17c45fc9fddd075d86ca1b0" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 41065 } } }
I20260811 04:47:59.954339 14819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0 [sys.catalog]: This master's current role is: LEADER
I20260811 04:47:59.954246 14829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:47:59.957844 14829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:47:59.963452 14829 catalog_manager.cc:1347] Generated new cluster ID: e78e9cab500d4fa8b5c06126a1d167e6
I20260811 04:47:59.963529 14829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:47:59.970955 14829 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:47:59.971334 14829 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:47:59.983472 14829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19b62c84b17c45fc9fddd075d86ca1b0: Generated new TSK 0
I20260811 04:47:59.984277 14829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 04:47:59.988806 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14728
2026-08-11T04:47:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1558 ms)
[----------] 18 tests from AuthTokenIssuingTest (28197 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:48:00 dist-test-slave-ss4n krb5kdc[14842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:48:00 dist-test-slave-ss4n krb5kdc[14842](info): set up 2 sockets
Aug 11 04:48:00 dist-test-slave-ss4n krb5kdc[14842](info): commencing operation
krb5kdc: starting...
W20260811 04:48:01.182219 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:48:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:48:01.325148 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:32791
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:32791,127.22.144.189:39933,127.22.144.188:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:48:01.443539 14856 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 04:48:01.443907 14856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:48:01.444021 14856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:48:01.444119 14856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:48:01.448292 14856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:48:01.448392 14856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:48:01.448432 14856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:48:01.448469 14856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:48:01.448508 14856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:48:01.453984 14856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:32791,127.22.144.189:39933,127.22.144.188:37481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:32791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:48:01.455112 14856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:48:01.456179 14856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:48:01.465245 14863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:01.465190 14862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:01.465196 14865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:01.465515 14856 server_base.cc:1034] running on GCE node
I20260811 04:48:01.466090 14856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:48:01.467222 14856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:48:01.468437 14856 hybrid_clock.cc:648] HybridClock initialized: now 1786423681468425 us; error 35 us; skew 500 ppm
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:48:01.471470 14856 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:48:01.472487 14856 webserver.cc:460] Webserver started at http://127.22.144.190:44873/ using document root <none> and password file <none>
I20260811 04:48:01.473075 14856 fs_manager.cc:359] Metadata directory not provided
I20260811 04:48:01.473183 14856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:48:01.473425 14856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:48:01.475605 14856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "036c595667cb45efbc98e42008d86fe7"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "4223f4d0eeee86004b407d949698674f"
server_key_iv: "368d861f4956468f6e3c511d40d12d45"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.476218 14856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "036c595667cb45efbc98e42008d86fe7"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "4223f4d0eeee86004b407d949698674f"
server_key_iv: "368d861f4956468f6e3c511d40d12d45"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.480079 14856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 04:48:01.482720 14873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:48:01.483902 14856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 04:48:01.484045 14856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "036c595667cb45efbc98e42008d86fe7"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "4223f4d0eeee86004b407d949698674f"
server_key_iv: "368d861f4956468f6e3c511d40d12d45"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.484187 14856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:48:01.517228 14856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:48:01.520924 14856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:48:01.521188 14856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:48:01.529279 14856 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:32791
I20260811 04:48:01.529639 14931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:32791 every 8 connection(s)
I20260811 04:48:01.530483 14856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:48:01.533571 14932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:48:01.533576 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14856
I20260811 04:48:01.533715 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:48:01.534018 23106 external_mini_cluster.cc:1428] Setting key 6809defac4c4ac2a616a57bebcb24d65
I20260811 04:48:01.538281 14932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.144.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.189@KRBTEST.COM" created.
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:48:01.555280 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.540806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48492 (local address 127.22.144.190:32791)
0811 04:48:01.541263 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:01.541283 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:01.541596 (+ 313us) server_negotiation.cc:407] Connection header received
0811 04:48:01.542690 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:01.542694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:01.542869 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:01.543078 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:01.543929 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.544968 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.546109 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.546709 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.546982 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.550057 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:01.550069 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:01.550086 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:01.550121 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:01.551929 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.552449 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.552454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.552455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.552569 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.552921 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.552924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.552926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.553338 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:01.553486 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:01.553907 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 04:48:01.554179 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":304,"thread_start_us":153,"threads_started":1}
I20260811 04:48:01.555280 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.540808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:32791 (local address 127.0.0.1:48492)
0811 04:48:01.541410 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.541447 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:01.542375 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:01.543068 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:01.543074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:01.543502 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:01.543761 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.543775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.545119 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.545122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.545974 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.545985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.546868 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.546871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.547024 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.547711 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:01.547738 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:01.549839 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:01.552058 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.552069 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.552080 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.552323 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.552666 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.552670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.552672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.552819 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.553456 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:01.553463 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:01.553652 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 04:48:01.553828 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 04:48:01.554230 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":107,"spinlock_wait_cycles":12544,"thread_start_us":159,"threads_started":1}
I20260811 04:48:01.558647 14932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:01.559600 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.559250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58706)
0811 04:48:01.559417 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.559516 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 04:48:01.559808 14875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
I20260811 04:48:01.560879 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.560530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58708)
0811 04:48:01.560668 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.560756 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 04:48:01.561144 14932 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:39933: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab.
Entry for principal kudu/127.22.144.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab.
I20260811 04:48:01.567080 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:39933
--webserver_interface=127.22.144.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab
--principal=kudu/127.22.144.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:32791,127.22.144.189:39933,127.22.144.188:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:48:01.610445 14932 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } attempt: 1
I20260811 04:48:01.611577 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.611174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58710)
0811 04:48:01.611378 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.611460 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 04:48:01.612506 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.612176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58716)
0811 04:48:01.612300 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.612378 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 04:48:01.612644 14932 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:39933: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
I20260811 04:48:01.680858 14932 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } attempt: 2
I20260811 04:48:01.681886 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.681532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58720)
0811 04:48:01.681700 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.681784 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 04:48:01.682914 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.682559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58734)
0811 04:48:01.682743 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.682822 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 04:48:01.683200 14932 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:39933: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39933: connect: Connection refused (error 111)
W20260811 04:48:01.687026 14939 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 04:48:01.687410 14939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:48:01.687480 14939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:48:01.687534 14939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:48:01.691983 14939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:48:01.692057 14939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:48:01.692077 14939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:48:01.692095 14939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:48:01.692111 14939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:48:01.698146 14939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:32791,127.22.144.189:39933,127.22.144.188:37481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab
--principal=kudu/127.22.144.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:39933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.144.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:48:01.699386 14939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:48:01.700552 14939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:48:01.709762 14946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:01.709748 14944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:01.709748 14948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:01.710115 14939 server_base.cc:1034] running on GCE node
I20260811 04:48:01.710870 14939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:48:01.712061 14939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:48:01.713267 14939 hybrid_clock.cc:648] HybridClock initialized: now 1786423681713253 us; error 40 us; skew 500 ppm
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:48:01.716495 14939 init.cc:377] Logged in from keytab as kudu/127.22.144.189@KRBTEST.COM (short username kudu)
I20260811 04:48:01.717581 14939 webserver.cc:460] Webserver started at http://127.22.144.189:40561/ using document root <none> and password file <none>
I20260811 04:48:01.718163 14939 fs_manager.cc:359] Metadata directory not provided
I20260811 04:48:01.718255 14939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:48:01.718501 14939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:48:01.720629 14939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data/instance:
uuid: "3f82f5e598064a0f905783b647452784"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "d7b0d498af3ae478068d9043a46ddc57"
server_key_iv: "fd9e96704fc612d194a941cfe1a794b7"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.721395 14939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal/instance:
uuid: "3f82f5e598064a0f905783b647452784"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "d7b0d498af3ae478068d9043a46ddc57"
server_key_iv: "fd9e96704fc612d194a941cfe1a794b7"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.725312 14939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 04:48:01.727921 14956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:48:01.729151 14939 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 04:48:01.729308 14939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal
uuid: "3f82f5e598064a0f905783b647452784"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "d7b0d498af3ae478068d9043a46ddc57"
server_key_iv: "fd9e96704fc612d194a941cfe1a794b7"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.729435 14939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:48:01.755663 14939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:48:01.759348 14939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:48:01.759583 14939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:48:01.767597 14939 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.189:39933
I20260811 04:48:01.767933 15016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.189:39933 every 8 connection(s)
I20260811 04:48:01.768699 14939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
I20260811 04:48:01.771883 15017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:48:01.774433 14932 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } attempt: 3
I20260811 04:48:01.775478 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 14939
I20260811 04:48:01.775650 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-1/wal/instance
I20260811 04:48:01.775985 23106 external_mini_cluster.cc:1428] Setting key fd9afeb28510ce522ca7ba698e47f67d
I20260811 04:48:01.776732 15017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.188@KRBTEST.COM; defaulting to no policy
I20260811 04:48:01.790702 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.774994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58738)
0811 04:48:01.775162 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.775182 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:01.775291 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:01.778895 (+ 3604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:01.778898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:01.779126 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:01.779269 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.779278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.781075 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.781087 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.781830 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.781839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.782664 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.782667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.782769 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.783937 (+ 1168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:01.783949 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:01.785870 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:01.788335 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.788342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.788347 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.788670 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.789049 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.789052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.789053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.789152 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.790074 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:01.790077 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:01.790194 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 04:48:01.790305 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 04:48:01.790439 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.188@KRBTEST.COM" created.
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:48:01.792135 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.775912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.22.144.189:39933)
0811 04:48:01.776511 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:01.776535 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:01.776586 (+ 51us) server_negotiation.cc:407] Connection header received
0811 04:48:01.778006 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:01.778038 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:01.778577 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:01.778953 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:01.779446 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.780872 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.781960 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.782530 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.782710 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.786041 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:01.786047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:01.786058 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:01.786086 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:01.788165 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.788794 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.788800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.788805 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.788921 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.789278 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.789281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.789283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.789932 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:01.790061 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:01.790459 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 04:48:01.791036 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":189,"threads_started":1}
I20260811 04:48:01.794675 14932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:01.795536 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.795212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58556)
0811 04:48:01.795365 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.795433 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 04:48:01.796320 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.796023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58562)
0811 04:48:01.796200 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.796251 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 04:48:01.796553 14932 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:37481: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
I20260811 04:48:01.796818 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.780376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.22.144.190:32791)
0811 04:48:01.780565 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:01.780574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:01.781105 (+ 531us) server_negotiation.cc:407] Connection header received
0811 04:48:01.781352 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:01.781355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:01.781411 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:01.781535 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:01.783304 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.783792 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.785351 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.785744 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.785859 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.792625 (+ 6766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:01.792632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:01.792638 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:01.792671 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:01.794593 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.795208 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.795211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.795213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.795332 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.795766 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.795769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.795771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.796007 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:01.796068 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:01.796554 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 04:48:01.796678 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 04:48:01.796857 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.780241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:32791 (local address 127.0.0.1:48498)
0811 04:48:01.780926 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.780994 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:01.781226 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:01.781642 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:01.781654 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:01.782814 (+ 1160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:01.783075 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.783087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.783953 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.783959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.785186 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.785204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.785953 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.785961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.786130 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.786949 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:01.786970 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:01.792464 (+ 5494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:01.794713 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.794724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.794730 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.795061 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.795447 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.795450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.795452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.795546 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.796182 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:01.796188 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:01.796320 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 04:48:01.796482 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 04:48:01.796679 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":153,"threads_started":1}
I20260811 04:48:01.798686 15017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
I20260811 04:48:01.808728 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.799299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:58754)
0811 04:48:01.799499 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.799516 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:01.799655 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:01.799935 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:01.799939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:01.800156 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:01.800267 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.800274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.801402 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.801405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.802182 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:01.802194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.802823 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.802826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:01.802913 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.803455 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:01.803467 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:01.805147 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:01.807026 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.807028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.807030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.807304 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.807689 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:01.807692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:01.807694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:01.807809 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:01.808278 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:01.808280 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:01.808362 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 04:48:01.808484 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 04:48:01.808598 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 04:48:01.809007 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.799405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.22.144.189:39933)
0811 04:48:01.799519 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:01.799524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:01.799566 (+ 42us) server_negotiation.cc:407] Connection header received
0811 04:48:01.799769 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:01.799774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:01.799821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:01.799890 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:01.800418 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.801268 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.802329 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:01.802675 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:01.802767 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:01.805340 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:01.805343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:01.805345 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:01.805367 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:01.806921 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.807423 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.807426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.807427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.807522 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:01.807956 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:01.807960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:01.807961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:01.808178 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:01.808325 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:01.808599 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 04:48:01.808751 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 04:48:01.809871 15017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:01.810729 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.810381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58564)
0811 04:48:01.810558 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.810669 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 04:48:01.811385 14958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
I20260811 04:48:01.812356 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.812086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58568)
0811 04:48:01.812231 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.812288 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab.
Entry for principal kudu/127.22.144.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab.
W20260811 04:48:01.812580 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:37481: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
I20260811 04:48:01.813678 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:37481
--webserver_interface=127.22.144.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab
--principal=kudu/127.22.144.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:32791,127.22.144.189:39933,127.22.144.188:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:48:01.827736 14932 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } attempt: 1
I20260811 04:48:01.828743 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.828350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58580)
0811 04:48:01.828593 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.828660 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 04:48:01.829756 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.829341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58584)
0811 04:48:01.829610 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.829684 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 04:48:01.830031 14932 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:37481: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
I20260811 04:48:01.861900 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } attempt: 1
I20260811 04:48:01.862855 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.862503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58586)
0811 04:48:01.862685 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.862761 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 04:48:01.863668 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.863424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:58590)
0811 04:48:01.863540 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.863604 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 04:48:01.863828 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:37481: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
I20260811 04:48:01.905294 14932 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } attempt: 2
I20260811 04:48:01.906512 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.906051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60630)
0811 04:48:01.906256 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.906349 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 04:48:01.907361 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.907118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60640)
0811 04:48:01.907239 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.907300 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 04:48:01.907505 14932 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:37481: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
I20260811 04:48:01.932011 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } attempt: 2
I20260811 04:48:01.932978 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.932611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60652)
0811 04:48:01.932775 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.932843 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 04:48:01.933811 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:01.933584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60668)
0811 04:48:01.933700 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:01.933756 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 04:48:01.933964 15017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:37481: Network error: Client connection negotiation failed: client connection to 127.22.144.188:37481: connect: Connection refused (error 111)
W20260811 04:48:01.937654 15025 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 04:48:01.937950 15025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:48:01.938004 15025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:48:01.938134 15025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:48:01.942474 15025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:48:01.942543 15025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:48:01.942564 15025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:48:01.942581 15025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:48:01.942621 15025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:48:01.948280 15025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:32791,127.22.144.189:39933,127.22.144.188:37481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab
--principal=kudu/127.22.144.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.144.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:48:01.949553 15025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:48:01.950773 15025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:48:01.960474 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:01.960611 15030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:01.960474 15031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:01.960701 15025 server_base.cc:1034] running on GCE node
I20260811 04:48:01.961386 15025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:48:01.962713 15025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:48:01.963958 15025 hybrid_clock.cc:648] HybridClock initialized: now 1786423681963941 us; error 38 us; skew 500 ppm
Aug 11 04:48:01 dist-test-slave-ss4n krb5kdc[14842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:48:01.967176 15025 init.cc:377] Logged in from keytab as kudu/127.22.144.188@KRBTEST.COM (short username kudu)
I20260811 04:48:01.968232 15025 webserver.cc:460] Webserver started at http://127.22.144.188:40655/ using document root <none> and password file <none>
I20260811 04:48:01.968798 15025 fs_manager.cc:359] Metadata directory not provided
I20260811 04:48:01.968863 15025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:48:01.969043 15025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:48:01.971367 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data/instance:
uuid: "f6d208276298456293afbd4dcdafbb44"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "dc27bc8ad22cb6dcc9e66555c843f8c7"
server_key_iv: "0d05aec49b4c1d7bd5e7690b8c002fad"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.971909 15025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal/instance:
uuid: "f6d208276298456293afbd4dcdafbb44"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "dc27bc8ad22cb6dcc9e66555c843f8c7"
server_key_iv: "0d05aec49b4c1d7bd5e7690b8c002fad"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.976016 15025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 04:48:01.978844 15041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:48:01.980451 15025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:48:01.980623 15025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal
uuid: "f6d208276298456293afbd4dcdafbb44"
format_stamp: "Formatted at 2026-08-11 04:48:01 on dist-test-slave-ss4n"
server_key: "dc27bc8ad22cb6dcc9e66555c843f8c7"
server_key_iv: "0d05aec49b4c1d7bd5e7690b8c002fad"
server_key_version: "encryptionkey@0"
I20260811 04:48:01.980767 15025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:48:01.991766 15025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:48:01.995399 15025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:48:01.995645 15025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:48:02.003746 15025 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.188:37481
I20260811 04:48:02.004058 15099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.188:37481 every 8 connection(s)
I20260811 04:48:02.004921 15025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
I20260811 04:48:02.006705 14932 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } attempt: 3
I20260811 04:48:02.008270 15101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:48:02.011667 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 15025
I20260811 04:48:02.011837 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786423274535345-23106-0/minicluster-data/master-2/wal/instance
I20260811 04:48:02.012171 23106 external_mini_cluster.cc:1428] Setting key f60d96a0f8069cf6e3cc4f7fe269d2ed
I20260811 04:48:02.013571 15101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:48:02.021919 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.007293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60676)
0811 04:48:02.007467 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:02.007486 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:02.007595 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:02.010266 (+ 2671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:02.010273 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:02.010500 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:02.010634 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.010643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.012705 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.012709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.013425 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.013441 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.014500 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.014505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.014600 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.015218 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:02.015242 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:02.016803 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:02.019507 (+ 2704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.019512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.019514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.019801 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.020164 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.020168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.020170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.020267 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.021146 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:02.021153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:02.021304 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 04:48:02.021580 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 04:48:02.021763 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:48:02.023289 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.008322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.22.144.188:37481)
0811 04:48:02.008776 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.008791 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.008830 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:48:02.009562 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.009581 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.009927 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.010238 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.010858 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.012538 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.013583 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.014345 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.014546 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.017012 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.017017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.017031 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.017059 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.019372 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.019918 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.019925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.019939 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.020060 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.020374 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.020377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.020378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.020998 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.021146 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.021730 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.022093 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":308,"thread_start_us":148,"threads_started":1}
I20260811 04:48:02.025136 15017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } attempt: 3
I20260811 04:48:02.027835 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.013311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.22.144.188:37481)
0811 04:48:02.013668 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.013678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.013692 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:48:02.013729 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.013732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.013777 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.013874 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.014941 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.015625 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.016697 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.017026 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.017575 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.020249 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.020252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.020255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.020279 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.024309 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.024902 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.024906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.024908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.024996 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.025844 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.025850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.025855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.026095 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.027390 (+ 1295us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.027460 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.027596 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":285,"thread_start_us":181,"threads_started":1}
I20260811 04:48:02.028380 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.015887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.22.144.190:32791)
0811 04:48:02.016020 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.016024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.017105 (+ 1081us) server_negotiation.cc:407] Connection header received
0811 04:48:02.017243 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.017246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.017302 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.017396 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.018366 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.018905 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.020244 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.020541 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.020673 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.024184 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.024188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.024192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.024228 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.026032 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.026724 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.026729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.026732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.026824 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.027154 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.027158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.027159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.027459 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.027532 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.028070 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.028195 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 04:48:02.028400 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.015826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:32791 (local address 127.0.0.1:48314)
0811 04:48:02.016950 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:02.017011 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:02.017148 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:02.017461 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:02.017470 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:02.018068 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:02.018234 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.018245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.019078 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.019084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.020097 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.020110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.020691 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.020695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.020786 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.021447 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:02.021466 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:02.024004 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:02.026280 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.026290 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.026296 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.026591 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.026951 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.026954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.026956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.027044 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.027613 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:02.027618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:02.027810 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 04:48:02.028004 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 04:48:02.028195 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":26,"thread_start_us":122,"threads_started":1}
I20260811 04:48:02.029428 14932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7: Bootstrap starting.
I20260811 04:48:02.029448 15101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:48:02.033731 14932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7: Neither blocks nor log segments found. Creating new log.
I20260811 04:48:02.034879 14932 log.cc:824] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:48:02.037613 14932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7: No bootstrap required, opened a new log
I20260811 04:48:02.038091 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.025675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60700)
0811 04:48:02.025823 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:02.025840 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:02.025932 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:02.027192 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:02.027196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:02.027396 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:02.027516 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.027523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.029723 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.029728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.030586 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.030595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.031285 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.031289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.031385 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.031991 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:02.032007 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:02.033666 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:02.036072 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.036077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.036079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.036370 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.036737 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.036741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.036744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.036844 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.037442 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:02.037447 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:02.037592 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 04:48:02.037699 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 04:48:02.037862 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 04:48:02.039055 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.026206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.22.144.188:37481)
0811 04:48:02.026965 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.026969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.026980 (+ 11us) server_negotiation.cc:407] Connection header received
0811 04:48:02.027012 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.027015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.027054 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.027144 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.027683 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.029438 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.030753 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.031131 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.031671 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.033819 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.033823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.033826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.033850 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.035938 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.036504 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.036509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.036512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.036607 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.037069 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.037075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.037079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.037318 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.038591 (+ 1273us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.038700 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.038844 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":673,"spinlock_wait_cycles":2304}
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
I20260811 04:48:02.042706 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.030787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.22.144.190:32791)
0811 04:48:02.030894 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.030904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.030914 (+ 10us) server_negotiation.cc:407] Connection header received
0811 04:48:02.030941 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.030943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.030974 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.031066 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.031753 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.032313 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.033545 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.034093 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.034219 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.037773 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.037777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.037780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.037806 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.039578 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.040157 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.040160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.040162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.040531 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.041199 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.041202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.041204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.041446 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.041659 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.042421 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.042553 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 04:48:02.043218 15017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: Bootstrap starting.
I20260811 04:48:02.043357 14932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.043637 14932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 036c595667cb45efbc98e42008d86fe7, State: Initialized, Role: FOLLOWER
I20260811 04:48:02.043751 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.030392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.22.144.189:39933)
0811 04:48:02.030537 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.030543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.030557 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:48:02.030596 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.030600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.030655 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.030738 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.034471 (+ 3733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.035253 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.036508 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.036844 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.036932 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.039884 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.039888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.039891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.039929 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.041709 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.042278 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.042282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.042284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.042378 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.042709 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.042713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.042714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.042970 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.043063 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.043460 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.043588 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 04:48:02.044006 14932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.044256 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.030150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39933 (local address 127.0.0.1:34682)
0811 04:48:02.030398 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:02.030414 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:02.030485 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:02.033642 (+ 3157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:02.033645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:02.033829 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:02.033960 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.033967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.035417 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.035422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.036345 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.036358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.037128 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.037131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.037240 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.037775 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:02.037791 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:02.039671 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:02.041846 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.041850 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.041852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.042135 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.042499 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.042502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.042504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.042600 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.043142 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:02.043144 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:02.043251 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 04:48:02.043930 (+ 679us) client_negotiation.cc:240] Negotiation successful
0811 04:48:02.044102 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 04:48:02.045090 15101 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:02.045951 14932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:48:02.046394 15017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: Neither blocks nor log segments found. Creating new log.
I20260811 04:48:02.047328 15017 log.cc:824] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: Log is configured to *not* fsync() on all Append() calls
I20260811 04:48:02.048702 15109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.048898 15109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
I20260811 04:48:02.051569 15017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: No bootstrap required, opened a new log
Aug 11 04:48:02 dist-test-slave-ss4n krb5kdc[14842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423681, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:48:02.055287 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.044084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.22.144.189:39933)
0811 04:48:02.044199 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.044203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.044355 (+ 152us) server_negotiation.cc:407] Connection header received
0811 04:48:02.044532 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.044538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.044597 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.044655 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.045597 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.046326 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.047480 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.047814 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.047914 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.051473 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.051477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.051479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.051505 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.053237 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.053797 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.053801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.053803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.053890 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.054229 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.054234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.054236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.054462 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.054616 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.054978 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.055108 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 04:48:02.056202 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.045709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:37481 (local address 127.0.0.1:60704)
0811 04:48:02.046360 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:02.046378 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:02.046469 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:02.046800 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:02.046803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:02.047013 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:02.047152 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.047164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.048303 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.048306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.049144 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:02.049189 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.049935 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.049939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:02.050020 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.050494 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:02.050510 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:02.052187 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:02.054523 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.054528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.054532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.054835 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.055141 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:02.055144 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:02.055146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:02.055228 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:02.055662 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:02.055666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:02.055756 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 04:48:02.055859 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 04:48:02.056005 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":139,"threads_started":1}
I20260811 04:48:02.056764 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.046138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60704 (local address 127.22.144.188:37481)
0811 04:48:02.046260 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.046264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.046552 (+ 288us) server_negotiation.cc:407] Connection header received
0811 04:48:02.046581 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.046585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.046624 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.046699 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.047301 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.048176 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.049319 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.049714 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.049823 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.052349 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.052353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.052356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.052379 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.054382 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.054948 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.054951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.054953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.055032 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.055319 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.055322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.055324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.055539 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.055627 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.056486 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.056614 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:48:02.056808 15017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.057333 15017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f82f5e598064a0f905783b647452784, State: Initialized, Role: FOLLOWER
I20260811 04:48:02.057783 15017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.058337 15123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.058521 15123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:02.051940 14856 master_runner.cc:186] Error getting master registration for 127.22.144.189:39933: OK, instance_id { permanent_uuid: "3f82f5e598064a0f905783b647452784" instance_seqno: 1786423681765312 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 04:48:02.059506 15017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:48:02.059525 14856 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 04:48:02.059564 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:48:02.060256 15101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: Bootstrap starting.
I20260811 04:48:02.061244 14901 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "036c595667cb45efbc98e42008d86fe7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318
I20260811 04:48:02.061816 14901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 04:48:02.061967 14901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:48:02.063470 15101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: Neither blocks nor log segments found. Creating new log.
I20260811 04:48:02.063529 14901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.064426 15101 log.cc:824] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: Log is configured to *not* fsync() on all Append() calls
I20260811 04:48:02.065167 14986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "036c595667cb45efbc98e42008d86fe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3f82f5e598064a0f905783b647452784"
I20260811 04:48:02.065399 14986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:48:02.065599 14901 leader_election.cc:288] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [CANDIDATE]: Term 1 election: Requested vote from peers 3f82f5e598064a0f905783b647452784 (127.22.144.189:39933), f6d208276298456293afbd4dcdafbb44 (127.22.144.188:37481)
W20260811 04:48:02.066179 15137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:48:02.066259 15137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:48:02.067546 14986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 036c595667cb45efbc98e42008d86fe7 in term 1.
I20260811 04:48:02.069330 14875 leader_election.cc:302] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 036c595667cb45efbc98e42008d86fe7, 3f82f5e598064a0f905783b647452784; no voters:
I20260811 04:48:02.069715 15109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:48:02.069957 15109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [term 1 LEADER]: Becoming Leader. State: Replica: 036c595667cb45efbc98e42008d86fe7, State: Running, Role: LEADER
I20260811 04:48:02.070338 15109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
W20260811 04:48:02.070519 15122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:48:02.070601 15122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:48:02.072355 15109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 036c595667cb45efbc98e42008d86fe7. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.072494 15109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:48:02.072978 15152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:48:02.072892 15101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: No bootstrap required, opened a new log
I20260811 04:48:02.070112 14939 master_runner.cc:186] Error getting master registration for 127.22.144.188:37481: OK, instance_id { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" instance_seqno: 1786423682001368 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 04:48:02.073635 14939 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 04:48:02.074120 15152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:48:02.085774 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.067905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.22.144.190:32791)
0811 04:48:02.068024 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.068029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.074118 (+ 6089us) server_negotiation.cc:407] Connection header received
0811 04:48:02.074574 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.074579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.074644 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.074722 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.075918 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.076762 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.078403 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.078765 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.078866 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.081345 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.081349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.081352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.081385 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.083328 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.083891 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.083895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.083896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.083996 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.084338 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.084341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.084343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.084749 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.084877 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.085383 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.085579 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 04:48:02.092181 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.068967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.22.144.188:37481)
0811 04:48:02.073769 (+ 4802us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.073775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.079761 (+ 5986us) server_negotiation.cc:407] Connection header received
0811 04:48:02.079949 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.079953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.080011 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.080066 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.081016 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.081818 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.083063 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.083414 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.083494 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.085410 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.085414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.085417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.085450 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.088771 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.089372 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.089376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.089377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.089552 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.091136 (+ 1584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.091140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.091142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.091380 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.091486 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.091859 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.091993 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313}
I20260811 04:48:02.091779 15101 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.093508 15101 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6d208276298456293afbd4dcdafbb44, State: Initialized, Role: FOLLOWER
I20260811 04:48:02.095039 14986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [term 1 FOLLOWER]: Refusing update from remote peer 036c595667cb45efbc98e42008d86fe7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:48:02.095902 15109 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:48:02.097543 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:02.069036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.22.144.189:39933)
0811 04:48:02.069155 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.069160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.078238 (+ 9078us) server_negotiation.cc:407] Connection header received
0811 04:48:02.078385 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.078388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.078454 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.078556 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.086448 (+ 7892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.087261 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.088653 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:02.089012 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:02.089110 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:02.093698 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:02.093703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:02.093707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:02.093740 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:02.095572 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.096121 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.096125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.096128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.096217 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:02.096512 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:02.096516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:02.096517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:02.096743 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:02.096856 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:02.097217 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 04:48:02.097342 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 04:48:02.094985 15101 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } }
I20260811 04:48:02.101948 15070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "036c595667cb45efbc98e42008d86fe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f6d208276298456293afbd4dcdafbb44"
I20260811 04:48:02.102201 15067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:48:02.101912 15101 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:48:02.104036 15067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [term 1 FOLLOWER]: Refusing update from remote peer 036c595667cb45efbc98e42008d86fe7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:48:02.105042 15109 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:48:02.105567 15070 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 036c595667cb45efbc98e42008d86fe7 in term 1.
I20260811 04:48:02.112020 15123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 036c595667cb45efbc98e42008d86fe7. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.112195 15123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:02.113142 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.113310 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:02.113446 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 036c595667cb45efbc98e42008d86fe7. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.113569 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:02.122246 15150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.122524 15150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:48:02.122720 15153 mvcc.cc:205] Tried to move back new op lower bound from 7317191401848119296 to 7317191401763958784. Current Snapshot: MvccSnapshot[applied={T|T < 7317191401848119296}]
I20260811 04:48:02.123395 15154 mvcc.cc:205] Tried to move back new op lower bound from 7317191401848119296 to 7317191401763958784. Current Snapshot: MvccSnapshot[applied={T|T < 7317191401848119296}]
I20260811 04:48:02.123584 15123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.123694 15123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:02.124635 15150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.124764 15150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7 [sys.catalog]: This master's current role is: LEADER
I20260811 04:48:02.127746 15169 mvcc.cc:205] Tried to move back new op lower bound from 7317191401848119296 to 7317191401763958784. Current Snapshot: MvccSnapshot[applied={T|T < 7317191401848119296 or (T in {7317191401848119296})}]
I20260811 04:48:02.128214 15155 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "036c595667cb45efbc98e42008d86fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "036c595667cb45efbc98e42008d86fe7" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 32791 } } peers { permanent_uuid: "3f82f5e598064a0f905783b647452784" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39933 } } peers { permanent_uuid: "f6d208276298456293afbd4dcdafbb44" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 37481 } } }
I20260811 04:48:02.128346 15155 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 04:48:02.129691 15172 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:48:02.129793 15172 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:48:02.143675 15152 catalog_manager.cc:1347] Generated new cluster ID: f3f422c55d45484d825f5a48873005a4
I20260811 04:48:02.143795 15152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:48:02.183907 15152 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:48:02.185591 15152 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:48:02.216773 15152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 036c595667cb45efbc98e42008d86fe7: Generated new TSK 0
I20260811 04:48:02.217895 15152 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 04:48:02.258114 14935 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.254447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.22.144.190:32791)
0811 04:48:02.254594 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.254600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.256767 (+ 2167us) server_negotiation.cc:407] Connection header received
0811 04:48:02.257002 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.257006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.257066 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.257128 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.257960 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48340: BlockingRecv error: recv got EOF from 127.0.0.1:48340 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 04:48:02.259697 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.255401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.22.144.189:39933)
0811 04:48:02.255545 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.255550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.258676 (+ 3126us) server_negotiation.cc:407] Connection header received
0811 04:48:02.258872 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.258876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.258939 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.258997 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.259559 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34704: BlockingRecv error: recv got EOF from 127.0.0.1:34704 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 04:48:02.261194 15105 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.256112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60716 (local address 127.22.144.188:37481)
0811 04:48:02.256236 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.256241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.260223 (+ 3982us) server_negotiation.cc:407] Connection header received
0811 04:48:02.260407 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.260410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.260485 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.260539 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.261065 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60716: BlockingRecv error: recv got EOF from 127.0.0.1:60716 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 04:48:02.267356 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.265315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.22.144.189:39933)
0811 04:48:02.265443 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.265448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.266159 (+ 711us) server_negotiation.cc:407] Connection header received
0811 04:48:02.266427 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.266434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.266522 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.266589 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.267243 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34716: BlockingRecv error: recv got EOF from 127.0.0.1:34716 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 04:48:02.273434 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.271623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.22.144.189:39933)
0811 04:48:02.271737 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.271741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.272334 (+ 593us) server_negotiation.cc:407] Connection header received
0811 04:48:02.272568 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.272572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.272634 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.272699 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.273313 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34726: BlockingRecv error: recv got EOF from 127.0.0.1:34726 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 04:48:02.280023 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.278238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.22.144.189:39933)
0811 04:48:02.278360 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.278365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.278930 (+ 565us) server_negotiation.cc:407] Connection header received
0811 04:48:02.279146 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.279150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.279207 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.279263 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.279893 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34732: BlockingRecv error: recv got EOF from 127.0.0.1:34732 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 04:48:02.289033 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.286957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.22.144.189:39933)
0811 04:48:02.287191 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.287196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.287864 (+ 668us) server_negotiation.cc:407] Connection header received
0811 04:48:02.288104 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.288109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.288183 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.288246 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.288902 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34746: BlockingRecv error: recv got EOF from 127.0.0.1:34746 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 04:48:02.302419 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.300090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.22.144.189:39933)
0811 04:48:02.300237 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.300242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.301087 (+ 845us) server_negotiation.cc:407] Connection header received
0811 04:48:02.301325 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.301329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.301387 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.301447 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.302292 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34762: BlockingRecv error: recv got EOF from 127.0.0.1:34762 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 04:48:02.323339 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.321826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.22.144.189:39933)
0811 04:48:02.321972 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.321976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.322305 (+ 329us) server_negotiation.cc:407] Connection header received
0811 04:48:02.322504 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.322514 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.322570 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.322678 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.323207 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34764: BlockingRecv error: recv got EOF from 127.0.0.1:34764 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 04:48:02.359440 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.358042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.22.144.189:39933)
0811 04:48:02.358221 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.358227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.358406 (+ 179us) server_negotiation.cc:407] Connection header received
0811 04:48:02.358594 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.358597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.358654 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.358753 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.359279 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34774: BlockingRecv error: recv got EOF from 127.0.0.1:34774 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 04:48:02.427225 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.426040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.22.144.189:39933)
0811 04:48:02.426321 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.426325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.426344 (+ 19us) server_negotiation.cc:407] Connection header received
0811 04:48:02.426402 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.426405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.426454 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.426595 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.427090 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34786: BlockingRecv error: recv got EOF from 127.0.0.1:34786 (error 108)
Metrics: {"server-negotiator.queue_time_us":179}
W20260811 04:48:02.558796 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.557652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.22.144.189:39933)
0811 04:48:02.557818 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.557826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.557937 (+ 111us) server_negotiation.cc:407] Connection header received
0811 04:48:02.558058 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.558060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.558113 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.558206 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.558635 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34802: BlockingRecv error: recv got EOF from 127.0.0.1:34802 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 04:48:02.818780 15019 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:02.817508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.22.144.189:39933)
0811 04:48:02.817680 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:02.817693 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:02.817801 (+ 108us) server_negotiation.cc:407] Connection header received
0811 04:48:02.817997 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:02.818001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:02.818064 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:02.818164 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:02.818668 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34814: BlockingRecv error: recv got EOF from 127.0.0.1:34814 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 04:48:03.067600 15137 catalog_manager.cc:1259] Loaded cluster ID: f3f422c55d45484d825f5a48873005a4
I20260811 04:48:03.067689 15137 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: loading cluster ID for follower catalog manager: success
I20260811 04:48:03.069371 15137 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: acquiring CA information for follower catalog manager: success
I20260811 04:48:03.070619 15137 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f82f5e598064a0f905783b647452784: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:48:03.131011 15172 catalog_manager.cc:1259] Loaded cluster ID: f3f422c55d45484d825f5a48873005a4
I20260811 04:48:03.131102 15172 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: loading cluster ID for follower catalog manager: success
I20260811 04:48:03.132893 15172 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: acquiring CA information for follower catalog manager: success
I20260811 04:48:03.134299 15172 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6d208276298456293afbd4dcdafbb44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:48:03.338033 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:03.333352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.22.144.189:39933)
0811 04:48:03.333678 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:03.333682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:03.333852 (+ 170us) server_negotiation.cc:407] Connection header received
0811 04:48:03.334067 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:03.334070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:03.334229 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:03.334336 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:03.334744 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.335289 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.336299 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.336544 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.336627 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:03.336976 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:03.337446 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:03.337727 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:03.337777 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 04:48:03.337855 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":151,"threads_started":1}
I20260811 04:48:03.345430 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:03.341447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60732 (local address 127.22.144.188:37481)
0811 04:48:03.341754 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:03.341758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:03.341772 (+ 14us) server_negotiation.cc:407] Connection header received
0811 04:48:03.341855 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:03.341858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:03.341982 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:03.342093 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:03.342395 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.342938 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.343836 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.344083 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.344183 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:03.344523 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:03.345032 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:03.345142 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:03.345242 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 04:48:03.345297 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":158,"threads_started":1}
I20260811 04:48:03.353883 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:03.349564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.22.144.189:39933)
0811 04:48:03.349701 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:03.349705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:03.349879 (+ 174us) server_negotiation.cc:407] Connection header received
0811 04:48:03.350141 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:03.350144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:03.350201 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:03.350250 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:03.350638 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.351390 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.352448 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.352691 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.352807 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:03.353097 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:03.353177 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:03.353263 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:03.353679 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 04:48:03.353741 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 04:48:03.354559 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:03.349184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.22.144.190:32791)
0811 04:48:03.349584 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:03.349591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:03.349736 (+ 145us) server_negotiation.cc:407] Connection header received
0811 04:48:03.349867 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:03.349870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:03.350068 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:03.350213 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:03.350490 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.351362 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.352916 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.353274 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.353372 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:03.353745 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:03.354137 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:03.354237 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:03.354312 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 04:48:03.354379 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":83,"threads_started":1}
I20260811 04:48:03.354692 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:03.349961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.22.144.188:37481)
0811 04:48:03.350091 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:03.350107 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:03.350120 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:48:03.350525 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:03.350528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:03.350593 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:03.350672 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:03.350919 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.351452 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.352916 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:03.353274 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:03.353372 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:03.353825 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:03.353911 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:03.354132 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:03.354508 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 04:48:03.354563 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 04:48:03.357931 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14856
I20260811 04:48:03.368623 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 14939
I20260811 04:48:03.379601 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 15025
2026-08-11T04:48:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3392 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 04:48:03 dist-test-slave-ss4n krb5kdc[15269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 04:48:03 dist-test-slave-ss4n krb5kdc[15269](info): set up 2 sockets
Aug 11 04:48:03 dist-test-slave-ss4n krb5kdc[15269](info): commencing operation
krb5kdc: starting...
W20260811 04:48:04.506768 23106 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/test-user.keytab.
Aug 11 04:48:04 dist-test-slave-ss4n krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T04:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T04:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.144.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
Entry for principal kudu/127.22.144.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab.
I20260811 04:48:04.649215 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:43333
--webserver_interface=127.22.144.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:43333,127.22.144.189:39621,127.22.144.188:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:48:04.767686 15283 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 04:48:04.768076 15283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:48:04.768157 15283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:48:04.768239 15283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:48:04.772289 15283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:48:04.772396 15283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:48:04.772434 15283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:48:04.772480 15283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:48:04.772513 15283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:48:04.778016 15283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:43333,127.22.144.189:39621,127.22.144.188:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.190.keytab
--principal=kudu/127.22.144.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.190:43333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.144.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:48:04.779224 15283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:48:04.780326 15283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:48:04.789592 15291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:04.789696 15283 server_base.cc:1034] running on GCE node
W20260811 04:48:04.789592 15289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:04.789635 15288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:04.791556 15283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:48:04.792836 15283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:48:04.794042 15283 hybrid_clock.cc:648] HybridClock initialized: now 1786423684794025 us; error 33 us; skew 500 ppm
Aug 11 04:48:04 dist-test-slave-ss4n krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:48:04.802286 15283 init.cc:377] Logged in from keytab as kudu/127.22.144.190@KRBTEST.COM (short username kudu)
I20260811 04:48:04.803368 15283 webserver.cc:460] Webserver started at http://127.22.144.190:34703/ using document root <none> and password file <none>
I20260811 04:48:04.804046 15283 fs_manager.cc:359] Metadata directory not provided
I20260811 04:48:04.804148 15283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:48:04.804399 15283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:48:04.806723 15283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data/instance:
uuid: "276ca520d82d452eb05175f1cac36f47"
format_stamp: "Formatted at 2026-08-11 04:48:04 on dist-test-slave-ss4n"
server_key: "808ecf89b2f348b830702756f0cc27c7"
server_key_iv: "6d481fb0bf62d8a14a6fd192607dc34a"
server_key_version: "encryptionkey@0"
I20260811 04:48:04.807438 15283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal/instance:
uuid: "276ca520d82d452eb05175f1cac36f47"
format_stamp: "Formatted at 2026-08-11 04:48:04 on dist-test-slave-ss4n"
server_key: "808ecf89b2f348b830702756f0cc27c7"
server_key_iv: "6d481fb0bf62d8a14a6fd192607dc34a"
server_key_version: "encryptionkey@0"
I20260811 04:48:04.811625 15283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:48:04.814407 15302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:48:04.815686 15283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:48:04.815843 15283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal
uuid: "276ca520d82d452eb05175f1cac36f47"
format_stamp: "Formatted at 2026-08-11 04:48:04 on dist-test-slave-ss4n"
server_key: "808ecf89b2f348b830702756f0cc27c7"
server_key_iv: "6d481fb0bf62d8a14a6fd192607dc34a"
server_key_version: "encryptionkey@0"
I20260811 04:48:04.815970 15283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:48:04.827219 15283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:48:04.830572 15283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:48:04.830811 15283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:48:04.838660 15283 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.190:43333
I20260811 04:48:04.838981 15359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.190:43333 every 8 connection(s)
I20260811 04:48:04.839862 15283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/data/info.pb
I20260811 04:48:04.843000 15360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:48:04.847070 15360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:04.848858 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 15283
I20260811 04:48:04.849018 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-0/wal/instance
I20260811 04:48:04.849340 23106 external_mini_cluster.cc:1428] Setting key aaa4e5a398d962921a5a0d7cdae60ded
Aug 11 04:48:04 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.144.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.189@KRBTEST.COM" created.
I20260811 04:48:04.865643 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.849614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.22.144.190:43333)
0811 04:48:04.850283 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:04.850302 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:04.850478 (+ 176us) server_negotiation.cc:407] Connection header received
0811 04:48:04.852039 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:04.852047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:04.852330 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:04.852548 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:04.853560 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:04.855057 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:04.856106 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:04.856818 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:04.857006 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:04.859873 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:04.859879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:04.859892 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:04.859915 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:04.861872 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:04.862487 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:04.862519 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:04.862523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:04.862698 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:04.863220 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:04.863223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:04.863224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:04.863705 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:04.863840 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:04.864383 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 04:48:04.864662 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":540,"thread_start_us":205,"threads_started":1}
I20260811 04:48:04.865648 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.849614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:43333 (local address 127.0.0.1:48446)
0811 04:48:04.850282 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.850344 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:04.851673 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:04.852555 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:04.852559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:04.853077 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:04.853361 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:04.853376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:04.855209 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:04.855211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:04.855968 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:04.855978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:04.856996 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:04.856999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:04.857084 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:04.857830 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:04.857848 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:04.859703 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:04.862034 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:04.862043 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:04.862052 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:04.862322 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:04.862880 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:04.862884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:04.862886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:04.863064 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:04.863857 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:04.863867 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:04.864068 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 04:48:04.864287 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 04:48:04.864773 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":111,"spinlock_wait_cycles":30976,"thread_start_us":171,"threads_started":1}
I20260811 04:48:04.868757 15360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:04.869665 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.869318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42008)
0811 04:48:04.869448 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.869588 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 04:48:04.869864 15304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
I20260811 04:48:04.870734 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.870517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42016)
0811 04:48:04.870630 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.870682 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 04:48:04.870900 15360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:39621: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab.
Entry for principal kudu/127.22.144.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab.
I20260811 04:48:04.885329 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:39621
--webserver_interface=127.22.144.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab
--principal=kudu/127.22.144.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:43333,127.22.144.189:39621,127.22.144.188:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 04:48:04.920142 15360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } attempt: 1
I20260811 04:48:04.921103 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.920726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42026)
0811 04:48:04.920881 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.921020 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 04:48:04.921936 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.921710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42028)
0811 04:48:04.921824 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.921881 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 04:48:04.922137 15360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:39621: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
I20260811 04:48:04.990370 15360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } attempt: 2
I20260811 04:48:04.991401 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.991062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42040)
0811 04:48:04.991246 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.991320 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 04:48:04.992316 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:04.992074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42048)
0811 04:48:04.992210 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:04.992260 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 04:48:04.992570 15360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.189:39621: Network error: Client connection negotiation failed: client connection to 127.22.144.189:39621: connect: Connection refused (error 111)
W20260811 04:48:05.004834 15367 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 04:48:05.005244 15367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:48:05.005311 15367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:48:05.005388 15367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 04:48:05.009995 15367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:48:05.010095 15367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:48:05.010167 15367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:48:05.010219 15367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:48:05.010257 15367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:48:05.016062 15367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:43333,127.22.144.189:39621,127.22.144.188:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.189.keytab
--principal=kudu/127.22.144.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.189:39621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.144.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:48:05.017314 15367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:48:05.018487 15367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:48:05.028080 15373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:05.028282 15367 server_base.cc:1034] running on GCE node
W20260811 04:48:05.028101 15375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:05.028080 15372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:05.029003 15367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:48:05.030270 15367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:48:05.031441 15367 hybrid_clock.cc:648] HybridClock initialized: now 1786423685031428 us; error 32 us; skew 500 ppm
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:48:05.034497 15367 init.cc:377] Logged in from keytab as kudu/127.22.144.189@KRBTEST.COM (short username kudu)
I20260811 04:48:05.035609 15367 webserver.cc:460] Webserver started at http://127.22.144.189:36869/ using document root <none> and password file <none>
I20260811 04:48:05.036216 15367 fs_manager.cc:359] Metadata directory not provided
I20260811 04:48:05.036283 15367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:48:05.036509 15367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:48:05.038763 15367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data/instance:
uuid: "bea55144ecb8411da6754e5d9ccf4ef5"
format_stamp: "Formatted at 2026-08-11 04:48:05 on dist-test-slave-ss4n"
server_key: "dbc96dfd04867d7c7a0df4d54475c584"
server_key_iv: "2820b154f027c6a66561c7b91cf9dddf"
server_key_version: "encryptionkey@0"
I20260811 04:48:05.039377 15367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal/instance:
uuid: "bea55144ecb8411da6754e5d9ccf4ef5"
format_stamp: "Formatted at 2026-08-11 04:48:05 on dist-test-slave-ss4n"
server_key: "dbc96dfd04867d7c7a0df4d54475c584"
server_key_iv: "2820b154f027c6a66561c7b91cf9dddf"
server_key_version: "encryptionkey@0"
I20260811 04:48:05.043565 15367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 04:48:05.046581 15383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:48:05.047935 15367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 04:48:05.048060 15367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal
uuid: "bea55144ecb8411da6754e5d9ccf4ef5"
format_stamp: "Formatted at 2026-08-11 04:48:05 on dist-test-slave-ss4n"
server_key: "dbc96dfd04867d7c7a0df4d54475c584"
server_key_iv: "2820b154f027c6a66561c7b91cf9dddf"
server_key_version: "encryptionkey@0"
I20260811 04:48:05.048223 15367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:48:05.064817 15367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:48:05.068740 15367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:48:05.069000 15367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:48:05.077455 15367 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.189:39621
I20260811 04:48:05.077891 15441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.189:39621 every 8 connection(s)
I20260811 04:48:05.078723 15367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/data/info.pb
I20260811 04:48:05.081853 15442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:48:05.083627 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 15367
I20260811 04:48:05.083788 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-1/wal/instance
I20260811 04:48:05.083788 15360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } attempt: 3
I20260811 04:48:05.084152 23106 external_mini_cluster.cc:1428] Setting key f1e347d72eac57565027deff6e5fefae
I20260811 04:48:05.088729 15442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.144.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.144.188@KRBTEST.COM" created.
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:48:05.105077 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.084428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42064)
0811 04:48:05.084665 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.084684 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.084787 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.087774 (+ 2987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.087778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.088024 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.088210 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.088220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.092135 (+ 3915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.092141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.093077 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.093090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.095547 (+ 2457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.095552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.095692 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.096333 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.096355 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.100210 (+ 3855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.103080 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.103083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.103085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.103284 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.103667 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.103673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.103678 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.103761 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.104532 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.104535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.104643 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.104753 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.104905 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 04:48:05.105427 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.091129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.22.144.190:43333)
0811 04:48:05.091269 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.091280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.091816 (+ 536us) server_negotiation.cc:407] Connection header received
0811 04:48:05.092098 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.092105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.092190 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.092312 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.093259 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.094198 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.095236 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.096561 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.096686 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.100964 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.100968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.100971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.101004 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.102814 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.103517 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.103520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.103522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.103603 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.104027 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.104031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.104032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.104259 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.104357 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.104969 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.105135 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:48:05.106451 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.085215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.22.144.189:39621)
0811 04:48:05.085690 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.085719 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.085757 (+ 38us) server_negotiation.cc:407] Connection header received
0811 04:48:05.086786 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.086819 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.087347 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.087828 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.089599 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.092005 (+ 2406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.094378 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.095393 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.095614 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.100375 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.100383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.100398 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.100426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.102968 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.103397 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.103404 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.103405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.103546 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.103857 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.103860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.103861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.104416 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.104553 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.104856 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.105459 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":320,"thread_start_us":162,"threads_started":1}
I20260811 04:48:05.106456 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.091105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:43333 (local address 127.0.0.1:48448)
0811 04:48:05.091676 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.091732 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.091922 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.092387 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.092394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.092913 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.093097 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.093105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.094348 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.094351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.095094 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.095105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.096750 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.096754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.096843 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.097462 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.097612 (+ 150us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.100814 (+ 3202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.102963 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.102977 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.102994 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.103375 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.103721 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.103728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.103733 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.103909 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.104380 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.104392 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.104603 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.105647 (+ 1044us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.105889 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":44,"thread_start_us":171,"threads_started":1}
I20260811 04:48:05.108294 15442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:05.108767 15360 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:05.109592 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.109270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59570)
0811 04:48:05.109410 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.109513 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 04:48:05.110607 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.110287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59580)
0811 04:48:05.110469 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.110524 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260811 04:48:05.110751 15360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:34895: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
I20260811 04:48:05.118247 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.108908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42068)
0811 04:48:05.109068 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.109082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.109176 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.109522 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.109525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.109720 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.109810 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.109816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.110619 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.110625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.111241 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.111251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.111769 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.111771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.111893 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.112400 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.112412 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.114198 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.116458 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.116460 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.116462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.116801 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.117086 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.117088 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.117089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.117179 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.117765 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.117767 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.117862 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.117969 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.118080 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 04:48:05.118319 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.108971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.22.144.189:39621)
0811 04:48:05.109069 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.109074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.109124 (+ 50us) server_negotiation.cc:407] Connection header received
0811 04:48:05.109272 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.109276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.109317 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.109389 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.109931 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.110502 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.111372 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.111657 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.111759 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.114401 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.114405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.114409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.114465 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.116350 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.116902 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.116905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.116906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.116995 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.117309 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.117316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.117321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.117675 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.117764 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.118053 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.118160 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 04:48:05.119443 15442 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:05.120316 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.119983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59588)
0811 04:48:05.120098 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.120244 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.144.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab.
Entry for principal kudu/127.22.144.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab.
W20260811 04:48:05.120534 15387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
I20260811 04:48:05.121579 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.121339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59594)
0811 04:48:05.121448 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.121517 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 04:48:05.121699 23106 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
/tmp/dist-test-taskZWTiLN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:34895
--webserver_interface=127.22.144.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab
--principal=kudu/127.22.144.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.144.148:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.144.190:43333,127.22.144.189:39621,127.22.144.188:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 04:48:05.121742 15442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:34895: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
I20260811 04:48:05.141916 15360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } attempt: 1
I20260811 04:48:05.142905 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.142549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59596)
0811 04:48:05.142736 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.142810 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 04:48:05.143716 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.143473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59600)
0811 04:48:05.143594 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.143654 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 04:48:05.143874 15360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:34895: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
I20260811 04:48:05.171049 15442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } attempt: 1
I20260811 04:48:05.172057 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.171680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59602)
0811 04:48:05.171884 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.171962 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 04:48:05.172881 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.172634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59608)
0811 04:48:05.172761 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.172817 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 04:48:05.173046 15442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:34895: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
I20260811 04:48:05.219055 15360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } attempt: 2
I20260811 04:48:05.220039 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.219684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59610)
0811 04:48:05.219865 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.219943 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 04:48:05.220990 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.220727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59618)
0811 04:48:05.220861 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.220922 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 04:48:05.221165 15360 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:34895: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
W20260811 04:48:05.239794 15450 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 04:48:05.240178 15450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 04:48:05.240245 15450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 04:48:05.240326 15450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 04:48:05.241225 15442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } attempt: 2
I20260811 04:48:05.242131 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.241795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59624)
0811 04:48:05.241972 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.242043 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 04:48:05.242950 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.242722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59638)
0811 04:48:05.242834 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.242886 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 04:48:05.243141 15442 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.144.188:34895: Network error: Client connection negotiation failed: client connection to 127.22.144.188:34895: connect: Connection refused (error 111)
W20260811 04:48:05.245180 15450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 04:48:05.245242 15450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 04:48:05.245263 15450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 04:48:05.245280 15450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 04:48:05.245332 15450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 04:48:05.250993 15450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.144.148:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.144.190:43333,127.22.144.189:39621,127.22.144.188:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/krb5kdc/kudu_127.22.144.188.keytab
--principal=kudu/127.22.144.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.144.188:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.144.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 03:56:35 UTC on ccycloud.quasar-iyycfb.root.comops.site
build id 4579566
I20260811 04:48:05.252292 15450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 04:48:05.253471 15450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 04:48:05.262490 15455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:05.262576 15458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 04:48:05.262502 15456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 04:48:05.262864 15450 server_base.cc:1034] running on GCE node
I20260811 04:48:05.263379 15450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 04:48:05.264537 15450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 04:48:05.265699 15450 hybrid_clock.cc:648] HybridClock initialized: now 1786423685265679 us; error 32 us; skew 500 ppm
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 04:48:05.268455 15450 init.cc:377] Logged in from keytab as kudu/127.22.144.188@KRBTEST.COM (short username kudu)
I20260811 04:48:05.269443 15450 webserver.cc:460] Webserver started at http://127.22.144.188:35895/ using document root <none> and password file <none>
I20260811 04:48:05.270025 15450 fs_manager.cc:359] Metadata directory not provided
I20260811 04:48:05.270090 15450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 04:48:05.270265 15450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 04:48:05.272277 15450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data/instance:
uuid: "6c91dd29355740998c2e3b7deda7d7ef"
format_stamp: "Formatted at 2026-08-11 04:48:05 on dist-test-slave-ss4n"
server_key: "3c3a375ec417a5dddad2dfdeb61b7033"
server_key_iv: "acb93ace94bec5bd0498e27bbff59dc8"
server_key_version: "encryptionkey@0"
I20260811 04:48:05.272781 15450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal/instance:
uuid: "6c91dd29355740998c2e3b7deda7d7ef"
format_stamp: "Formatted at 2026-08-11 04:48:05 on dist-test-slave-ss4n"
server_key: "3c3a375ec417a5dddad2dfdeb61b7033"
server_key_iv: "acb93ace94bec5bd0498e27bbff59dc8"
server_key_version: "encryptionkey@0"
I20260811 04:48:05.276575 15450 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 04:48:05.279423 15465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 04:48:05.281021 15450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 04:48:05.281208 15450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data,/tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal
uuid: "6c91dd29355740998c2e3b7deda7d7ef"
format_stamp: "Formatted at 2026-08-11 04:48:05 on dist-test-slave-ss4n"
server_key: "3c3a375ec417a5dddad2dfdeb61b7033"
server_key_iv: "acb93ace94bec5bd0498e27bbff59dc8"
server_key_version: "encryptionkey@0"
I20260811 04:48:05.281350 15450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 04:48:05.294158 15450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 04:48:05.297814 15450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 04:48:05.298096 15450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 04:48:05.306851 15450 rpc_server.cc:307] RPC server started. Bound to: 127.22.144.188:34895
I20260811 04:48:05.307227 15527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.144.188:34895 every 8 connection(s)
I20260811 04:48:05.308101 15450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/data/info.pb
I20260811 04:48:05.309465 23106 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu as pid 15450
I20260811 04:48:05.309728 23106 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZWTiLN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786423274535345-23106-0/minicluster-data/master-2/wal/instance
I20260811 04:48:05.310072 23106 external_mini_cluster.cc:1428] Setting key 16101d74ee3d8ff7f0f8f5f49c315a19
I20260811 04:48:05.311836 15528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 04:48:05.316871 15528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:48:05.320333 15360 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } attempt: 3
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.190@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:48:05.327069 15532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.311578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.22.144.188:34895)
0811 04:48:05.311996 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.312023 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.312062 (+ 39us) server_negotiation.cc:407] Connection header received
0811 04:48:05.312757 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.312776 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.313129 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.313422 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.314573 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.315607 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.316918 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.317458 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.317695 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.320914 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.320921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.320935 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.320974 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.323152 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.323730 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.323737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.323745 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.323918 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.324287 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.324292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.324296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.324915 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.325165 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.325655 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.325946 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":276,"spinlock_wait_cycles":19712,"thread_start_us":151,"threads_started":1}
I20260811 04:48:05.330910 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.319211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.22.144.190:43333)
0811 04:48:05.319377 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.319382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.319800 (+ 418us) server_negotiation.cc:407] Connection header received
0811 04:48:05.319993 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.319997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.320107 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.320199 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.321348 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.322117 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.323287 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.323622 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.323736 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.326587 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.326593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.326598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.326631 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.328432 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.328992 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.328995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.328997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.329226 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.329572 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.329575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.329577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.329936 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.330062 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.330558 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.330711 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 04:48:05.331769 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.319176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.190:43333 (local address 127.0.0.1:48458)
0811 04:48:05.319668 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.319711 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.319873 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.320284 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.320290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.321023 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.321228 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.321238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.322297 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.322301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.323163 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.323170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.323760 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.323765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.323895 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.324474 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.324488 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.326368 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.328566 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.328579 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.328585 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.328864 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.329323 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.329326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.329328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.329428 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.330126 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.330133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.330280 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.331345 (+ 1065us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.331579 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"spinlock_wait_cycles":2688,"thread_start_us":163,"threads_started":1}
I20260811 04:48:05.332401 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.320784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59664)
0811 04:48:05.320980 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.320997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.321079 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.321659 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.321662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.321841 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.321930 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.321936 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.322980 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.322985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.323743 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.323756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.324564 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.324569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.324665 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.325172 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.325191 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.326814 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.330669 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.330676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.330679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.330931 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.331278 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.331281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.331284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.331386 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.331903 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.331906 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.332011 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.332088 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.332222 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 04:48:05.333020 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.320859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.22.144.188:34895)
0811 04:48:05.321354 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.321364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.321377 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:48:05.321414 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.321443 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.321531 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.321628 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.322059 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.322841 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.323896 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.324366 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.324563 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.326959 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.326962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.326965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.326987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.330553 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.331051 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.331057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.331061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.331169 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.331580 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.331583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.331585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.331788 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.332648 (+ 860us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.332720 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.332830 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":112,"threads_started":1}
I20260811 04:48:05.333159 15528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:05.334338 15442 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } attempt: 3
I20260811 04:48:05.336437 15360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47: Bootstrap starting.
I20260811 04:48:05.339586 15360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47: Neither blocks nor log segments found. Creating new log.
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.189@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.190@KRBTEST.COM
I20260811 04:48:05.340587 15360 log.cc:824] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
I20260811 04:48:05.345096 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.334766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59676)
0811 04:48:05.334991 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.335003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.335084 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.335329 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.335332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.335567 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.335660 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.335666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.336655 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.336659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.337402 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.337415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.338104 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.338108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.338195 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.338767 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.338782 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.340912 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.343383 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.343386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.343388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.343661 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.343989 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.343993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.343995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.344080 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.344563 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.344566 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.344672 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.344763 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.344912 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 04:48:05.345635 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.329825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.22.144.190:43333)
0811 04:48:05.330146 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.330161 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.330174 (+ 13us) server_negotiation.cc:407] Connection header received
0811 04:48:05.330208 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.330212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.330260 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.330363 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.331137 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.331889 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.332962 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.333296 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.333400 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.341373 (+ 7973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.341377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.341380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.341408 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.343412 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.343979 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.343982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.343984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.344069 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.344422 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.344425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.344427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.344653 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.344882 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.345150 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.345282 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":87,"threads_started":1}
I20260811 04:48:05.345819 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.334862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.22.144.188:34895)
0811 04:48:05.334998 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.335003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.335044 (+ 41us) server_negotiation.cc:407] Connection header received
0811 04:48:05.335169 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.335172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.335215 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.335300 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.335780 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.336498 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.337591 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.337960 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.338423 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.341079 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.341083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.341086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.341110 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.343242 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.343791 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.343794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.343796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.343880 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.344200 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.344203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.344205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.344448 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.345416 (+ 968us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.345523 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.345654 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 04:48:05.346621 15360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47: No bootstrap required, opened a new log
I20260811 04:48:05.350253 15442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: Bootstrap starting.
I20260811 04:48:05.351318 15360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.351601 15360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276ca520d82d452eb05175f1cac36f47, State: Initialized, Role: FOLLOWER
I20260811 04:48:05.352125 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.337747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.22.144.189:39621)
0811 04:48:05.337902 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.337908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.337925 (+ 17us) server_negotiation.cc:407] Connection header received
0811 04:48:05.337967 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.337971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.338069 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.338186 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.338872 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.339839 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.340863 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.341242 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.341329 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.344253 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.344257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.344260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.344286 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.347193 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.350515 (+ 3322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.350520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.350522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.350623 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.350963 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.350966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.350968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.351246 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.351324 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.351839 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.351973 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 04:48:05.352068 15360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.352957 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.333714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.189:39621 (local address 127.0.0.1:42082)
0811 04:48:05.333989 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.334005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.334083 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.338339 (+ 4256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.338343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.338549 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.338698 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.338710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.339968 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.339970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.340710 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.340724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.341534 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.341541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.341671 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.342328 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.342350 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.344069 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.349949 (+ 5880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.349953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.349955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.350289 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.350750 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.350753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.350755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.350857 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.351401 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.351403 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.351499 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.352605 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.352764 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 04:48:05.353516 15442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: Neither blocks nor log segments found. Creating new log.
I20260811 04:48:05.354070 15528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } has no permanent_uuid. Determining permanent_uuid...
I20260811 04:48:05.354146 15360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:48:05.355556 15442 log.cc:824] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: Log is configured to *not* fsync() on all Append() calls
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.144.189@KRBTEST.COM
Aug 11 04:48:05 dist-test-slave-ss4n krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786423685, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.144.188@KRBTEST.COM for kudu/127.22.144.188@KRBTEST.COM
I20260811 04:48:05.363152 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.347038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.22.144.189:39621)
0811 04:48:05.347386 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.347393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.349765 (+ 2372us) server_negotiation.cc:407] Connection header received
0811 04:48:05.349928 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.349932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.349981 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.350900 (+ 919us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.351004 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.351801 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.352906 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.353282 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.353810 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.356536 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.356540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.356543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.356569 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.360432 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.361017 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.361020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.361022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.361111 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.361570 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.361574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.361575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.361814 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.362722 (+ 908us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.362808 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.362944 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":131,"threads_started":1}
I20260811 04:48:05.360524 15283 master_runner.cc:186] Error getting master registration for 127.22.144.189:39621: OK, instance_id { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" instance_seqno: 1786423685075154 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 04:48:05.365371 15283 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 04:48:05.365669 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.354862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.22.144.188:34895)
0811 04:48:05.354963 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.354970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.354986 (+ 16us) server_negotiation.cc:407] Connection header received
0811 04:48:05.355046 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.355052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.355109 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.355204 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.356602 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.357322 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.358742 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.359073 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.359152 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.361852 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.361857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.361860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.361882 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.363803 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.364341 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.364344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.364346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.364428 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.364705 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.364708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.364710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.364916 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.364990 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.365337 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.365499 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 04:48:05.366192 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.354652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.144.188:34895 (local address 127.0.0.1:59682)
0811 04:48:05.354811 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 04:48:05.354828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 04:48:05.354919 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 04:48:05.356084 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 04:48:05.356090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 04:48:05.356328 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 04:48:05.356449 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.356458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.357430 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.357434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.358604 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 04:48:05.358616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.359305 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.359309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 04:48:05.359408 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.359920 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 04:48:05.359936 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 04:48:05.361704 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 04:48:05.363941 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.363944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.363946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.364225 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.364533 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 04:48:05.364536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 04:48:05.364539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 04:48:05.364617 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 04:48:05.365066 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 04:48:05.365069 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 04:48:05.365153 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 04:48:05.365871 (+ 718us) client_negotiation.cc:240] Negotiation successful
0811 04:48:05.366038 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 04:48:05.368531 15442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: No bootstrap required, opened a new log
I20260811 04:48:05.369642 15528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: Bootstrap starting.
I20260811 04:48:05.370075 23106 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 04:48:05.371492 15330 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "276ca520d82d452eb05175f1cac36f47"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48472
I20260811 04:48:05.371820 15330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 04:48:05.371954 15330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:48:05.373471 15330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.373764 15442 raft_consensus.cc:348] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.374029 15442 raft_consensus.cc:729] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea55144ecb8411da6754e5d9ccf4ef5, State: Initialized, Role: FOLLOWER
I20260811 04:48:05.374401 15442 consensus_queue.cc:260] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.375221 15528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: Neither blocks nor log segments found. Creating new log.
I20260811 04:48:05.375797 15330 leader_election.cc:288] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [CANDIDATE]: Term 1 election: Requested vote from peers bea55144ecb8411da6754e5d9ccf4ef5 (127.22.144.189:39621), 6c91dd29355740998c2e3b7deda7d7ef (127.22.144.188:34895)
I20260811 04:48:05.376029 15442 sys_catalog.cc:564] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:48:05.376083 15414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "276ca520d82d452eb05175f1cac36f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bea55144ecb8411da6754e5d9ccf4ef5"
I20260811 04:48:05.376305 15414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:48:05.376443 15528 log.cc:824] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: Log is configured to *not* fsync() on all Append() calls
I20260811 04:48:05.377729 15538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.377878 15538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:05.378484 15414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276ca520d82d452eb05175f1cac36f47 in term 1.
W20260811 04:48:05.378753 15552 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:48:05.378837 15552 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:48:05.379031 15304 leader_election.cc:302] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276ca520d82d452eb05175f1cac36f47, bea55144ecb8411da6754e5d9ccf4ef5; no voters:
I20260811 04:48:05.380049 15538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 04:48:05.380453 15538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [term 1 LEADER]: Becoming Leader. State: Replica: 276ca520d82d452eb05175f1cac36f47, State: Running, Role: LEADER
I20260811 04:48:05.381109 15538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
W20260811 04:48:05.383674 15566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:48:05.383775 15566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:48:05.384486 15538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 276ca520d82d452eb05175f1cac36f47. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.384630 15538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: This master's current role is: LEADER
I20260811 04:48:05.385262 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.385419 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:05.385751 15575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 04:48:05.387024 15575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 04:48:05.391564 15528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: No bootstrap required, opened a new log
I20260811 04:48:05.391026 15367 master_runner.cc:186] Error getting master registration for 127.22.144.188:34895: OK, instance_id { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" instance_seqno: 1786423685304405 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 04:48:05.395454 15367 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 04:48:05.396605 15528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.396945 15528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c91dd29355740998c2e3b7deda7d7ef, State: Initialized, Role: FOLLOWER
I20260811 04:48:05.397377 15528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } }
I20260811 04:48:05.399210 15528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 04:48:05.399925 15500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "276ca520d82d452eb05175f1cac36f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6c91dd29355740998c2e3b7deda7d7ef"
I20260811 04:48:05.400228 15500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 04:48:05.401999 15500 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276ca520d82d452eb05175f1cac36f47 in term 1.
I20260811 04:48:05.409940 15500 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [term 1 FOLLOWER]: Refusing update from remote peer 276ca520d82d452eb05175f1cac36f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:48:05.410743 15567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:48:05.412082 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.394396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.22.144.190:43333)
0811 04:48:05.394515 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.394519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.396143 (+ 1624us) server_negotiation.cc:407] Connection header received
0811 04:48:05.396529 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.396534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.396599 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.396664 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.398928 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.399782 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.404230 (+ 4448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.404638 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.404732 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.407053 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.407057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.407060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.407093 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.409840 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.410438 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.410443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.410447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.410569 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.410937 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.410940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.410943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.411203 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.411344 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.411783 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.411923 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 04:48:05.416263 15414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [term 1 FOLLOWER]: Refusing update from remote peer 276ca520d82d452eb05175f1cac36f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 04:48:05.416951 15567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 04:48:05.418329 15580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.418488 15580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:05.418614 15580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 276ca520d82d452eb05175f1cac36f47. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.418699 15580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:05.421387 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 276ca520d82d452eb05175f1cac36f47. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.421554 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 04:48:05.422339 15594 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 04:48:05.422430 15594 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 04:48:05.423023 15567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.423167 15567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: This master's current role is: LEADER
I20260811 04:48:05.423234 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.393945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.22.144.189:39621)
0811 04:48:05.394064 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.394069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.405778 (+ 11709us) server_negotiation.cc:407] Connection header received
0811 04:48:05.405999 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.406003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.406070 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.406124 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.408420 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.409163 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.412352 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.412738 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.412816 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.416857 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.416861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.416865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.416896 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.418665 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.419177 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.419181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.419184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.419269 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.421880 (+ 2611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.421883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.421885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.422142 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.422297 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.422912 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.423066 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 04:48:05.423938 15596 mvcc.cc:205] Tried to move back new op lower bound from 7317191415430938624 to 7317191415328411648. Current Snapshot: MvccSnapshot[applied={T|T < 7317191415430938624}]
I20260811 04:48:05.424288 15595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.424419 15595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:05.424587 15576 mvcc.cc:205] Tried to move back new op lower bound from 7317191415430938624 to 7317191415328411648. Current Snapshot: MvccSnapshot[applied={T|T < 7317191415430938624}]
I20260811 04:48:05.425978 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:05.395180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.22.144.188:34895)
0811 04:48:05.395315 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.395320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.407007 (+ 11687us) server_negotiation.cc:407] Connection header received
0811 04:48:05.407455 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.407459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.407510 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.407564 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.410376 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.411107 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.414749 (+ 3642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:05.415094 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:05.415168 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:05.419600 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 04:48:05.419605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 04:48:05.419609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 04:48:05.419649 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 04:48:05.421322 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.422628 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.422643 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.422645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.422732 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 04:48:05.424912 (+ 2180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 04:48:05.424916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 04:48:05.424917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 04:48:05.425139 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 04:48:05.425256 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:05.425706 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 04:48:05.425830 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 04:48:05.428967 15538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.429127 15538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47 [sys.catalog]: This master's current role is: LEADER
I20260811 04:48:05.432363 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "276ca520d82d452eb05175f1cac36f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "276ca520d82d452eb05175f1cac36f47" member_type: VOTER last_known_addr { host: "127.22.144.190" port: 43333 } } peers { permanent_uuid: "bea55144ecb8411da6754e5d9ccf4ef5" member_type: VOTER last_known_addr { host: "127.22.144.189" port: 39621 } } peers { permanent_uuid: "6c91dd29355740998c2e3b7deda7d7ef" member_type: VOTER last_known_addr { host: "127.22.144.188" port: 34895 } } }
I20260811 04:48:05.432509 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 04:48:05.432659 15575 catalog_manager.cc:1347] Generated new cluster ID: 24a75f357f9c40468a5ee4d07a583704
I20260811 04:48:05.432722 15575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 04:48:05.433677 15598 mvcc.cc:205] Tried to move back new op lower bound from 7317191415430938624 to 7317191415328411648. Current Snapshot: MvccSnapshot[applied={T|T < 7317191415430938624}]
I20260811 04:48:05.481927 15575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 04:48:05.483291 15575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 04:48:05.498900 15575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 276ca520d82d452eb05175f1cac36f47: Generated new TSK 0
I20260811 04:48:05.499869 15575 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 04:48:05.531559 15535 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.528155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48492 (local address 127.22.144.190:43333)
0811 04:48:05.528324 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.528329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.530291 (+ 1962us) server_negotiation.cc:407] Connection header received
0811 04:48:05.530576 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.530580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.530633 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.530687 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.531395 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48492: BlockingRecv error: recv got EOF from 127.0.0.1:48492 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 04:48:05.533277 15534 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.528886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.22.144.188:34895)
0811 04:48:05.529007 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.529011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.532000 (+ 2989us) server_negotiation.cc:407] Connection header received
0811 04:48:05.532385 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.532389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.532449 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.532503 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.533135 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59686: BlockingRecv error: recv got EOF from 127.0.0.1:59686 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 04:48:05.534617 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.529629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.22.144.189:39621)
0811 04:48:05.529748 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.529752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.533667 (+ 3915us) server_negotiation.cc:407] Connection header received
0811 04:48:05.533846 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.533850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.533900 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.533958 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.534485 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42106: BlockingRecv error: recv got EOF from 127.0.0.1:42106 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 04:48:05.539676 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.537370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.22.144.189:39621)
0811 04:48:05.537672 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.537677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.538580 (+ 903us) server_negotiation.cc:407] Connection header received
0811 04:48:05.538819 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.538824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.538910 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.538969 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.539555 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42112: BlockingRecv error: recv got EOF from 127.0.0.1:42112 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 04:48:05.545171 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.543491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.22.144.189:39621)
0811 04:48:05.543614 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.543618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.544168 (+ 550us) server_negotiation.cc:407] Connection header received
0811 04:48:05.544370 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.544373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.544426 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.544480 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.545046 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42124: BlockingRecv error: recv got EOF from 127.0.0.1:42124 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 04:48:05.551363 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.549689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.22.144.189:39621)
0811 04:48:05.549796 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.549801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.550363 (+ 562us) server_negotiation.cc:407] Connection header received
0811 04:48:05.550577 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.550581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.550637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.550691 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.551231 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42134: BlockingRecv error: recv got EOF from 127.0.0.1:42134 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 04:48:05.564242 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.562199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.22.144.189:39621)
0811 04:48:05.562383 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.562388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.563090 (+ 702us) server_negotiation.cc:407] Connection header received
0811 04:48:05.563343 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.563347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.563413 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.563470 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.564116 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42150: BlockingRecv error: recv got EOF from 127.0.0.1:42150 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260811 04:48:05.576794 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.575029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42156 (local address 127.22.144.189:39621)
0811 04:48:05.575155 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.575160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.575759 (+ 599us) server_negotiation.cc:407] Connection header received
0811 04:48:05.575964 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.575968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.576022 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.576076 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.576667 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42156: BlockingRecv error: recv got EOF from 127.0.0.1:42156 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 04:48:05.597427 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.595563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.22.144.189:39621)
0811 04:48:05.595698 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.595703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.596376 (+ 673us) server_negotiation.cc:407] Connection header received
0811 04:48:05.596627 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.596635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.596686 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.596781 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.597299 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42170: BlockingRecv error: recv got EOF from 127.0.0.1:42170 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 04:48:05.633193 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.631805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.22.144.189:39621)
0811 04:48:05.631944 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.631949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.632223 (+ 274us) server_negotiation.cc:407] Connection header received
0811 04:48:05.632402 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.632406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.632466 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.632563 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.633055 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42186: BlockingRecv error: recv got EOF from 127.0.0.1:42186 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 04:48:05.700989 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.699683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.22.144.189:39621)
0811 04:48:05.699907 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.699911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.700032 (+ 121us) server_negotiation.cc:407] Connection header received
0811 04:48:05.700167 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.700169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.700222 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.700342 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.700883 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42190: BlockingRecv error: recv got EOF from 127.0.0.1:42190 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
W20260811 04:48:05.833170 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:05.831786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.22.144.189:39621)
0811 04:48:05.831969 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:05.831974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:05.832128 (+ 154us) server_negotiation.cc:407] Connection header received
0811 04:48:05.832323 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:05.832329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:05.832388 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:05.832489 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:05.833015 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42202: BlockingRecv error: recv got EOF from 127.0.0.1:42202 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 04:48:06.092550 15537 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 04:48:06.091462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.22.144.189:39621)
0811 04:48:06.091600 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:06.091605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:06.091676 (+ 71us) server_negotiation.cc:407] Connection header received
0811 04:48:06.091833 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:06.091836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:06.091887 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:06.091971 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 04:48:06.092413 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42212: BlockingRecv error: recv got EOF from 127.0.0.1:42212 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 04:48:06.385229 15566 catalog_manager.cc:1259] Loaded cluster ID: 24a75f357f9c40468a5ee4d07a583704
I20260811 04:48:06.385318 15566 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: loading cluster ID for follower catalog manager: success
I20260811 04:48:06.387135 15566 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: acquiring CA information for follower catalog manager: success
I20260811 04:48:06.388363 15566 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bea55144ecb8411da6754e5d9ccf4ef5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:48:06.423867 15594 catalog_manager.cc:1259] Loaded cluster ID: 24a75f357f9c40468a5ee4d07a583704
I20260811 04:48:06.423959 15594 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: loading cluster ID for follower catalog manager: success
I20260811 04:48:06.425733 15594 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: acquiring CA information for follower catalog manager: success
I20260811 04:48:06.427013 15594 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6c91dd29355740998c2e3b7deda7d7ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 04:48:06.611389 15675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:06.607181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.22.144.189:39621)
0811 04:48:06.607477 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:06.607482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:06.607502 (+ 20us) server_negotiation.cc:407] Connection header received
0811 04:48:06.607560 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:06.607563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:06.607697 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:06.607811 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:06.608071 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.608641 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.609616 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.609880 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.609976 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:06.610311 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:06.610787 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:06.610942 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:06.611077 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 04:48:06.611159 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
I20260811 04:48:06.618613 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:06.614848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.22.144.188:34895)
0811 04:48:06.615104 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:06.615108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:06.615123 (+ 15us) server_negotiation.cc:407] Connection header received
0811 04:48:06.615280 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:06.615282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:06.615408 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:06.615528 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:06.615779 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.616310 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.617140 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.617359 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.617443 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:06.617791 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:06.618233 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:06.618349 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:06.618402 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 04:48:06.618453 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":98,"threads_started":1}
I20260811 04:48:06.626271 15675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:06.621993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.22.144.189:39621)
0811 04:48:06.622156 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:06.622163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:06.622440 (+ 277us) server_negotiation.cc:407] Connection header received
0811 04:48:06.622666 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:06.622672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:06.622743 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:06.622803 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:06.623280 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.624106 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.625072 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.625331 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.625450 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:06.625763 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:06.625842 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:06.625901 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:06.626045 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 04:48:06.626109 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 04:48:06.627152 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:06.622740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.22.144.188:34895)
0811 04:48:06.622849 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:06.622853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:06.623020 (+ 167us) server_negotiation.cc:407] Connection header received
0811 04:48:06.623188 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:06.623191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:06.623260 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:06.623383 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:06.623748 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.624315 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.625627 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.625917 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.626024 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:06.626721 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:06.626802 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:06.626876 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:06.626971 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 04:48:06.627024 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 04:48:06.627995 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 04:48:06.622082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.22.144.190:43333)
0811 04:48:06.622361 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 04:48:06.622367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 04:48:06.622693 (+ 326us) server_negotiation.cc:407] Connection header received
0811 04:48:06.622805 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 04:48:06.622809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 04:48:06.622980 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 04:48:06.623123 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 04:48:06.623378 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.624178 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.626090 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 04:48:06.626437 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 04:48:06.626555 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 04:48:06.626927 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 04:48:06.627347 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 04:48:06.627443 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 04:48:06.627759 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 04:48:06.627836 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":128,"threads_started":1}
I20260811 04:48:06.631316 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 15283
I20260811 04:48:06.641430 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 15367
I20260811 04:48:06.650480 23106 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZWTiLN/build/debug/bin/kudu with pid 15450
2026-08-11T04:48:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3271 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6664 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (412111 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
2 FAILED TESTS